-- Logs begin at Thu 2022-07-14 10:27:51 UTC, end at Thu 2022-07-14 11:35:30 UTC. -- Jul 14 10:52:31.083294 dv-cinder-848938-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 14 10:52:31.091303 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 14 10:52:31 2022] *** Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: nodename: dv-cinder-848938-2-iscsi Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: machine: x86_64 Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: clock source: unix Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: pcre jit disabled Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: detected number of CPU cores: 4 Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: current working directory: / Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: detected binary path: /usr/bin/uwsgi-core Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: your processes number limit is 47680 Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: your memory page size is 4096 bytes Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: detected max file descriptor number: 2048 Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: lock engine: pthread robust mutexes Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: thunder lock: enabled Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 14 10:52:31.096428 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 14 10:52:31.138007 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: Python main interpreter initialized at 0x563ff7fed9c0 Jul 14 10:52:31.138007 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: python threads support enabled Jul 14 10:52:31.138007 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: your server socket listen backlog is limited to 100 connections Jul 14 10:52:31.138007 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: your mercy for graceful operations on workers is 90 seconds Jul 14 10:52:31.138361 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: mapped 403077 bytes (393 KB) for 2 cores Jul 14 10:52:31.138436 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: *** Operational MODE: preforking *** Jul 14 10:52:31.138702 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: *** uWSGI is running in multiple interpreter mode *** Jul 14 10:52:31.138702 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: spawned uWSGI master process (pid: 80173) Jul 14 10:52:31.138908 dv-cinder-848938-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 14 10:52:31.140459 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: spawned uWSGI worker 1 (pid: 80182, cores: 1) Jul 14 10:52:31.140459 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: spawned uWSGI worker 2 (pid: 80183, cores: 1) Jul 14 10:52:31.140459 dv-cinder-848938-2-iscsi devstack@keystone.service[80173]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 14 10:52:36.665789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 10:52:36.665789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: from cryptography.utils import int_from_bytes Jul 14 10:52:36.666560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 10:52:36.666560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: from cryptography.utils import int_from_bytes Jul 14 10:52:36.666669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 10:52:36.666669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: from cryptography.utils import int_from_bytes Jul 14 10:52:36.666669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 10:52:36.666669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: from cryptography.utils import int_from_bytes Jul 14 10:52:36.715528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 10:52:36.715712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 10:52:36.715901 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] command line args: [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 10:52:36.716170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 10:52:36.716248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 10:52:36.716350 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ================================================================================ {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 10:52:36.716432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 10:52:36.716606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] command line args: [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 10:52:36.716686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] admin_token = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.716853 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] config_dir = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.716930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 10:52:36.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.717118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ================================================================================ {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 10:52:36.717222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] config_source = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.717306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] admin_token = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.717403 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] debug = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.717526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] config_dir = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.717609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.717844 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.717907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] config_source = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.717965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.718082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] debug = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.718168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.718270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.718399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.718507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.718615 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.718728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.718845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.718965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.719083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_config_append = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.719196 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.719318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.719318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_dir = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.719562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.719682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_file = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.719839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_config_append = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.719839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.720169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.720322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_dir = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.720456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.720595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_file = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.720720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.720850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.720969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.721094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.721228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.721344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.721480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.721626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.721767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.721877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.721953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.722095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.722196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.722282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.722373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.722456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.722543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.722621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.722761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.722862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.722944 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.723097 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.723260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.723365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.723529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.723619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.723619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.723817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.75/identity {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.723914 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.724002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] publish_errors = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.724002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.724232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.75/identity {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.724320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.724413 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] publish_errors = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.724523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.724634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.724717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.724855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.724934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.725040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.725127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] standard_threads = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.725224 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.725312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.725402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.725538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.725616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] standard_threads = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.725755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.725755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] use_journal = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.725961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.726082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] use_json = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.726215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.726309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] use_stderr = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.726382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.726472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] use_syslog = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.726554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] use_journal = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.726625 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.726725 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] use_json = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.726725 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] use_stderr = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.726855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.726954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] use_syslog = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.727027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.727125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:52:36.727202 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.727278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.727367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.727471 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.727543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.727641 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.727722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.727865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.727986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.728115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.728202 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.728357 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.728461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.728610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.728707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.728852 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.728941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.729097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.729187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.729346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.729459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.729584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.729671 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.729809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.729913 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.730057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.730163 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.730293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.730379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.730549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.730660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.730778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.730875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.731003 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.731088 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.731237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.731320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.731518 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.731618 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.731767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.731858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.732008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.732125 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.732240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.732316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.732474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.732571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.732706 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.732811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.732936 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.733039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.733172 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.733254 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.733423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.733571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.733720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.733800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.733893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.733981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.734099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.734195 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.734348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.734469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.734565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.734648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.734795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.734892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.735046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.735145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.735274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.735396 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.735538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.735659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.735772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.735854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.736001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.736185 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.736271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.736359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.736429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.736522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.736661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.736785 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.736906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.737003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.737116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.737210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.737346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.737436 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.737579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.737670 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.737810 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.737906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.738047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.738142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.738283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.738400 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.738535 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.738649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.738777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.738904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.739019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.739136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.739235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.739367 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.739570 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.739666 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.739770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.739888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.739978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.740064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.740155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.740258 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.740343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.740430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.740565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.740678 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.740825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.740945 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.741048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.741164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.741295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.741386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.741537 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.741650 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.741754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.741830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.741987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.742088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.742222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.742320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.742466 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.742572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.742705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.742830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.742944 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.743044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.743184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.743275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.743440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.743535 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.743677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.743757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.743911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.744032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.744151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.744269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.744389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.744515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.744627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.744752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.744972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.745090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.745215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.745292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.745443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.745539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.745677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.745761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.745917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.746028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.746134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.746232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.746375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.746464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.746617 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.746712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.746853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.746946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.747092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.747215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.747333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.747474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.747666 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.747761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.747851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.747931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.748052 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.748158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.748286 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.748386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.748490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.748575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.748722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.748821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.748958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.749057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.749194 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.749278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.749426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.749509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.749660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.749784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.749882 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.749979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.750119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.750217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.750356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.750453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.750607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.750692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.750849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.750974 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.751079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.external = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.751167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.751312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.751416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.751606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.751700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.751840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.external = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.751971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.752135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.password = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.752239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.752328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] auth.token = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.752422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.752500 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.752592 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.752739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.752856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.password = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.752972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.753074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] auth.token = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.753208 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.753290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.753454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.753563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.753702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.753787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.753932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.754015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.754167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.754271 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.754417 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.754537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.754635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.754728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.754875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.754969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.755120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.755201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.755365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.755570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.755687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.755774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.755874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.755949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.756091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.756174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.756850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.756967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.757120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.757206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.757378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.757503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.757615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.757722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.757870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.757968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.758114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.758197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.758372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.758478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.758611 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.758730 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.758844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.758922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.759078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.759175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.759323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.759436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.759580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.759763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.759874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.759965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.760079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.760196 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.760275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.760366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.760519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.760613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.760755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.760848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.760996 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.761080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.761235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.761357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.761478 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.761577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.761705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.761802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.761952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.762131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.762211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.762303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.762416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.762524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.762643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.762750 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.762876 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.762958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.763112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.763210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.763361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.763502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.763621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.763698 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.763843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.763925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.764077 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.764177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.764311 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.764396 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.764556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.764677 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.764783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.764862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.765005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.765102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.765253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.765338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.765500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.765600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.765752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.765873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.765983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.766060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.766214 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.766301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.766445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.766542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.766682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.766787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.766917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.767010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.767150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.767247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.767444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.767534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.767639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.767724 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.767869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.767977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.768102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.768184 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.768343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.768470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.768585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.768680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.768823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.768912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.769064 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.769147 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.769309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.769407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.769541 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.769628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.769778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.769911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.770012 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.770087 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.770227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.770308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.770464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.770564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.770692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.770789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.770921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.771047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.771154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.771237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.771404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.771503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.771647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.771734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.771877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.772011 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.772641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.772772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.772898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.772987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.773140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.773237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.773377 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.773462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.773612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.773731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.773846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.773928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.774077 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.774158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.774314 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.774393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.774551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.774641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.774787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.774874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.775025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.775110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.775273 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.775426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.775542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.775638 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.775776 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.775872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.776006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.776097 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.776242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.776340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.776478 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.776572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.776710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.776829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.776930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.777003 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.777170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.777262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.777406 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.777489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.777638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.777735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.777883 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.777988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.778128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.778207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.778361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.778472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.778603 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.778698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.778840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.778926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.779069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.779149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.779302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.779456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.779568 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.779655 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.779805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.779929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.780028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.780114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.780265 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.780348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.780513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.780618 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.780744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.780840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.780987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.781072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.781222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.781321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.781460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.781578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.781693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.781798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.781930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.782016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.782165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.782259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.782394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.782475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.782629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.782714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.782869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.782986 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.783108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.783196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.783325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.783428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.783591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.783703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.783843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.783941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.784094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.784194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.784329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.784444 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.784563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.784659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.784800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.784885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.785038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.785213 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.785293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] role.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.785387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.785507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] role.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.785623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.785742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.785832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.785977 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] role.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.786076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.786221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.786317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] role.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.786454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.786559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.786706 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.786830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.786954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.787038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.787199 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.787301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.787469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.787556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.787741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.787841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.787977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.788073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.788222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.788342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.788468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.788568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.788701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.788798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.788951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.789039 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.789195 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.789356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.789449 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.789591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.789685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.789845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.790074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.790160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.790309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.790393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.790559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.790644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.790791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.790873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.791045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.791157 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.791277 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.791362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.791540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.791641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.791779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.791866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.792016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.792103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.792245 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.792353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.792481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.792575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.792710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.792795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.792942 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.793033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.793196 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.793294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.793431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.793527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.793664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.793794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.793917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.793997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.794122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.794207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.794365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.794454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.794600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.794697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.794837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.794934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.795095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.795194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.795335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.795551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.795649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.795727 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.795839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.795935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.796076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.796174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.796314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.796397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.796552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.796675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.796798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.796890 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.797024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.797107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.797265 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.797358 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.797494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.797579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.797739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.797837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.797987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.798110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.798230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.798310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.connection = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.798461 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.798561 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.798742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.798835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.798984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.799082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.connection = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.799245 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.799355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.799523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.799654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.799763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.799859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.800002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.800086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.800237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.800318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.800477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.800587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.800718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.800802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.800940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.801022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.801173 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.801270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.801415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.801515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.801649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.801733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.801885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.802008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.802106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.802179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.802341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.802441 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.802578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.802671 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.802768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80183) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 10:52:36.802851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.803036 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.803297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:52:36.803492 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80182) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 10:52:36.833973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:36.834206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:36.834341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:36.834512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:36.835132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:36.835370 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:36.843486 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:36.843975 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:36.872295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:36.872522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:36.872688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:36.872922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.015493 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.015724 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.065200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.065381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.065491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.065572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.131152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.131328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.132076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.132222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.132370 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.132498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.132624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.132749 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.132881 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.133031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.133164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.133293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.157745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.157941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.158084 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.158210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.158335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.158460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.158584 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.158717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.158854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.158980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.159105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.159231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.159356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.159521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.170291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.170458 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.183896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.184103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.184258 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.184388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.184545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.184672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.184798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.184931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.185089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.185217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.185342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.185467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.185625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80182) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.185788 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80183) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:52:37.203159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.203372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 10:52:37.203535 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.203691 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 10:52:37.203966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.204172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 10:52:37.204326 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.204452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 10:52:37.204588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.204725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.204889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.205019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.205148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.205275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.205434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.205562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.216187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.216389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.216547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.216681 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.216830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.216977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.217108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.217239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.225849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.226043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.229177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.229458 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.229742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.230026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.230211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.230352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.230510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.230653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.230800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.230942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.231137 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.231294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.231474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.231623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.231770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.231923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.232077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.232161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.232399 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.232399 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.232687 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.232884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.260212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.261859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.262138 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.262653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.265287 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.265622 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.267501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.267785 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.268073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.271244 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.271764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.272292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.275644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.275934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.280463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.280572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.280751 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.280859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.281035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.281318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.281597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.281875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.282148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.282419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.285254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.285542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.285827 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.286114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.286393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.286671 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.286946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.287219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.304072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.304351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.307215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.307519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.309737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.312112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.313501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.313798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.315763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.316061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.319053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.321290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.322735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.324958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.326433 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.326735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.327040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.327328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.327641 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.327925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.328639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.328938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.329224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.329512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.329813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.330092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.347255 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.347576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.347850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.348117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.353290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.353791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.354083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.354358 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.357071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.358818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.359096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.362335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.362623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.362907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.363187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.363583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.364035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.366399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.366677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.369998 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.370281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.370563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.370844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.371122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.377901 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.378189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.378474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.378760 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.379043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.379338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.379641 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.384517 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.384802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.385082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.385362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.385642 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.385922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.386199 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.399164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.402524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.405583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.405874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.407526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.408878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.409303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.409596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.409887 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.412228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.414380 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.416196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.416482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.416772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.417820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.419623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.422894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.423187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.423513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.423802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.424169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.425798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.429607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.429898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.430182 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.430458 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.433918 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.434210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.434495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.439984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.440271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.440550 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.442422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80182) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.442704 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.442984 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.443261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.443559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.443866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.444145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.444422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.444794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.445071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.445362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.445628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.445909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.446180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80182) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.448493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80183) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:52:37.448781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.449065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.449343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.449613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.449884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.450171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.450440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.450807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.451090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.451371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.451659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.451928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.452205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80183) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:52:37.484790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563ff7fed9c0 pid: 80182 (default app) Jul 14 10:52:37.488251 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdb149-e2d5-4538-b486-f42816105364 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:37.488549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdb149-e2d5-4538-b486-f42816105364 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:37.488839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdb149-e2d5-4538-b486-f42816105364 None None] PATH_INFO: `/v3` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:37.490319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563ff7fed9c0 pid: 80183 (default app) Jul 14 10:52:37.493689 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1/1] 10.222.0.75 () {50 vars in 791 bytes} [Thu Jul 14 10:52:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:52:45.279694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56777265-7be5-4172-9416-647db123a0ce None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:45.280310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56777265-7be5-4172-9416-647db123a0ce None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:45.280531 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56777265-7be5-4172-9416-647db123a0ce None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:45.285578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1/2] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:52:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:52:45.291815 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f2dd6-c270-4d22-aef2-68bd5f512ced None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:45.292294 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f2dd6-c270-4d22-aef2-68bd5f512ced None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:45.293321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f2dd6-c270-4d22-aef2-68bd5f512ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:45.372438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: INFO dbcounter [None req-4e3f2dd6-c270-4d22-aef2-68bd5f512ced None None] Registered counter for database keystone Jul 14 10:52:45.394357 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG oslo_db.sqlalchemy.engines [None req-4e3f2dd6-c270-4d22-aef2-68bd5f512ced None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80182) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 10:52:45.401679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writer thread running {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 10:52:45.586574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG passlib.handlers.bcrypt [None req-4e3f2dd6-c270-4d22-aef2-68bd5f512ced None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80182) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 10:52:45.586900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG passlib.handlers.bcrypt [None req-4e3f2dd6-c270-4d22-aef2-68bd5f512ced None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80182) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 10:52:45.642296 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4e3f2dd6-c270-4d22-aef2-68bd5f512ced None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:45.643966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4e3f2dd6-c270-4d22-aef2-68bd5f512ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:45.770343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 2/3] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:52:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 14 10:52:45.992395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56777265-7be5-4172-9416-647db123a0ce None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:46.062938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: INFO dbcounter [None req-56777265-7be5-4172-9416-647db123a0ce None None] Registered counter for database keystone Jul 14 10:52:46.083700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG oslo_db.sqlalchemy.engines [None req-56777265-7be5-4172-9416-647db123a0ce None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80183) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 10:52:46.090434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writer thread running {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 10:52:46.214144 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56777265-7be5-4172-9416-647db123a0ce None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:46.218639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3ef4a-b974-45cf-b36b-a11c532e02d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:46.219404 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3ef4a-b974-45cf-b36b-a11c532e02d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:46.219690 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3ef4a-b974-45cf-b36b-a11c532e02d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:46.219972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3ef4a-b974-45cf-b36b-a11c532e02d6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:46.229261 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3ef4a-b974-45cf-b36b-a11c532e02d6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:46.286938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cff3ef4a-b974-45cf-b36b-a11c532e02d6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 10:52:46.288130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3ef4a-b974-45cf-b36b-a11c532e02d6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:46.298488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 2/4] 10.222.0.75 () {58 vars in 1324 bytes} [Thu Jul 14 10:52:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:52:46.305168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f2dd6-c270-4d22-aef2-68bd5f512ced None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:46.320126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f2dd6-c270-4d22-aef2-68bd5f512ced None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:46.325630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b4a64-887f-4de8-a145-50eb0c89299f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:46.327000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b4a64-887f-4de8-a145-50eb0c89299f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:46.327296 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b4a64-887f-4de8-a145-50eb0c89299f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:46.327615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b4a64-887f-4de8-a145-50eb0c89299f admin admin] PATH_INFO: `/v3/projects/75fd150c50ce43f4ac09652ce03bfda1` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:46.329526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b4a64-887f-4de8-a145-50eb0c89299f admin admin] RBAC: Authorizing `identity:get_project(project_id=75fd150c50ce43f4ac09652ce03bfda1)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:46.386918 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a43b4a64-887f-4de8-a145-50eb0c89299f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 10:52:46.387269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b4a64-887f-4de8-a145-50eb0c89299f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:46.392904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 3/5] 10.222.0.75 () {56 vars in 1207 bytes} [Thu Jul 14 10:52:46 2022] GET /identity/v3/projects/75fd150c50ce43f4ac09652ce03bfda1 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:52:48.541952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5d4f9-8453-45d8-af90-3341188b7abc None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:48.542454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5d4f9-8453-45d8-af90-3341188b7abc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:48.542454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5d4f9-8453-45d8-af90-3341188b7abc None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:48.546634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 3/6] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:52:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:52:48.552373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e113e-2279-4d7d-99e3-2b1ab1672942 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:48.552844 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e113e-2279-4d7d-99e3-2b1ab1672942 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:48.552969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e113e-2279-4d7d-99e3-2b1ab1672942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:48.572959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-249e113e-2279-4d7d-99e3-2b1ab1672942 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:48.574632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-249e113e-2279-4d7d-99e3-2b1ab1672942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:48.583113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 4/7] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:52:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 10:52:48.607377 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5d4f9-8453-45d8-af90-3341188b7abc None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:48.619695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5d4f9-8453-45d8-af90-3341188b7abc None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:48.623952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57daf80-95a5-46c3-b7d6-ab83d5742246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:48.624621 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57daf80-95a5-46c3-b7d6-ab83d5742246 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:48.624911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57daf80-95a5-46c3-b7d6-ab83d5742246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:48.625194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57daf80-95a5-46c3-b7d6-ab83d5742246 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:48.634249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57daf80-95a5-46c3-b7d6-ab83d5742246 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:48.636365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57daf80-95a5-46c3-b7d6-ab83d5742246 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:48.645469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 4/8] 10.222.0.75 () {58 vars in 1324 bytes} [Thu Jul 14 10:52:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:52:48.653291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e113e-2279-4d7d-99e3-2b1ab1672942 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:48.663098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e113e-2279-4d7d-99e3-2b1ab1672942 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:48.667713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80e6ba-dac7-47d4-8524-d19359d18137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:48.668600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80e6ba-dac7-47d4-8524-d19359d18137 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:48.668883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80e6ba-dac7-47d4-8524-d19359d18137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:48.669159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80e6ba-dac7-47d4-8524-d19359d18137 admin admin] PATH_INFO: `/v3/users/eae05ea06648449e9eda17d7c9a3f3fb` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:48.670589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80e6ba-dac7-47d4-8524-d19359d18137 admin admin] RBAC: Authorizing `identity:get_user(user_id=eae05ea06648449e9eda17d7c9a3f3fb)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:48.672490 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80e6ba-dac7-47d4-8524-d19359d18137 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:48.674320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 5/9] 10.222.0.75 () {56 vars in 1198 bytes} [Thu Jul 14 10:52:48 2022] GET /identity/v3/users/eae05ea06648449e9eda17d7c9a3f3fb => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:52:50.821688 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cf65f-35a1-4479-b224-c8e83abc8617 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:50.822214 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cf65f-35a1-4479-b224-c8e83abc8617 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:50.822214 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cf65f-35a1-4479-b224-c8e83abc8617 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:50.823621 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 5/10] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:52:50.829128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3329d7-36ca-4079-81f5-f97cb4ba513d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:50.829576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3329d7-36ca-4079-81f5-f97cb4ba513d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:50.829685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3329d7-36ca-4079-81f5-f97cb4ba513d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:50.831294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b82f29-8d46-412c-b422-93dd6e67545d None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:50.831616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b82f29-8d46-412c-b422-93dd6e67545d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:50.831898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b82f29-8d46-412c-b422-93dd6e67545d None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:50.833198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 6/11] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:52:50.838745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb6ad5-aa6f-4a9f-89e1-769f144d3940 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:50.839026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb6ad5-aa6f-4a9f-89e1-769f144d3940 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:50.839302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb6ad5-aa6f-4a9f-89e1-769f144d3940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:50.846803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fc3329d7-36ca-4079-81f5-f97cb4ba513d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:50.848075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fc3329d7-36ca-4079-81f5-f97cb4ba513d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:50.880505 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG passlib.handlers.bcrypt [None req-cafb6ad5-aa6f-4a9f-89e1-769f144d3940 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80183) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 10:52:50.880917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG passlib.handlers.bcrypt [None req-cafb6ad5-aa6f-4a9f-89e1-769f144d3940 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80183) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 10:52:50.926424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-cafb6ad5-aa6f-4a9f-89e1-769f144d3940 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:50.928671 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cafb6ad5-aa6f-4a9f-89e1-769f144d3940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:50.939406 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 6/12] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:52:50.962701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3329d7-36ca-4079-81f5-f97cb4ba513d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:50.987686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:51.036080 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 7/13] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:52:51.060298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb6ad5-aa6f-4a9f-89e1-769f144d3940 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:51.070584 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3329d7-36ca-4079-81f5-f97cb4ba513d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:51.105673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30daef-be7f-421d-b865-e8e7ec560793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:51.106249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30daef-be7f-421d-b865-e8e7ec560793 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.106535 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30daef-be7f-421d-b865-e8e7ec560793 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.106821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30daef-be7f-421d-b865-e8e7ec560793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.168228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb6ad5-aa6f-4a9f-89e1-769f144d3940 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:51.170553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:51.181060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30daef-be7f-421d-b865-e8e7ec560793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:51.181233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30daef-be7f-421d-b865-e8e7ec560793 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:51.210168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aebe6a-9e03-40d8-9c50-175d3a8c673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:51.210423 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:51.211298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aebe6a-9e03-40d8-9c50-175d3a8c673c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.211678 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aebe6a-9e03-40d8-9c50-175d3a8c673c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.211953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aebe6a-9e03-40d8-9c50-175d3a8c673c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.232649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aebe6a-9e03-40d8-9c50-175d3a8c673c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-60aebe6a-9e03-40d8-9c50-175d3a8c673c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:52:51.246022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:52:51.246980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:52:51.248056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 8/14] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:52:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:52:51.253221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aebe6a-9e03-40d8-9c50-175d3a8c673c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:51.293299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 7/15] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:52:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:52:51.298866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30daef-be7f-421d-b865-e8e7ec560793 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:51.376902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aebe6a-9e03-40d8-9c50-175d3a8c673c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:51.410374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30daef-be7f-421d-b865-e8e7ec560793 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:51.419710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751ed82-aa91-4d22-afca-641feec19363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:51.420545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751ed82-aa91-4d22-afca-641feec19363 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.420844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751ed82-aa91-4d22-afca-641feec19363 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.421119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751ed82-aa91-4d22-afca-641feec19363 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.421892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751ed82-aa91-4d22-afca-641feec19363 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:52:51.422200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751ed82-aa91-4d22-afca-641feec19363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:51.423701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751ed82-aa91-4d22-afca-641feec19363 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:51.436849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:51.438311 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 9/16] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:52:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:52:51.444166 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751ed82-aa91-4d22-afca-641feec19363 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:51.450973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6411c-3ffd-4a3f-b9c9-ba3d53db74d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:51.451811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6411c-3ffd-4a3f-b9c9-ba3d53db74d1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.452098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6411c-3ffd-4a3f-b9c9-ba3d53db74d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.452545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6411c-3ffd-4a3f-b9c9-ba3d53db74d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.463885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6411c-3ffd-4a3f-b9c9-ba3d53db74d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:51.465386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6411c-3ffd-4a3f-b9c9-ba3d53db74d1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:51.482827 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f7d6411c-3ffd-4a3f-b9c9-ba3d53db74d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:52:51.495753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:52:51.496634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:52:51.497437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:52:51.497437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:52:51.497437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:52:51.497437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:52:51.497437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 8/17] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:52:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:52:51.503571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6411c-3ffd-4a3f-b9c9-ba3d53db74d1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:51.558888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751ed82-aa91-4d22-afca-641feec19363 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:51.593815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b5c7f-d9ac-4add-aaed-21b57b348990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:51.594737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b5c7f-d9ac-4add-aaed-21b57b348990 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.595061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b5c7f-d9ac-4add-aaed-21b57b348990 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.595350 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b5c7f-d9ac-4add-aaed-21b57b348990 None admin] PATH_INFO: `/v3/users/eae05ea06648449e9eda17d7c9a3f3fb` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.596958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b5c7f-d9ac-4add-aaed-21b57b348990 None admin] RBAC: Authorizing `identity:get_user(user_id=eae05ea06648449e9eda17d7c9a3f3fb)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:51.598436 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b5c7f-d9ac-4add-aaed-21b57b348990 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:51.600128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 10/18] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:52:51 2022] GET /identity/v3/users/eae05ea06648449e9eda17d7c9a3f3fb => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:52:51.605657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b5c7f-d9ac-4add-aaed-21b57b348990 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:51.607885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6411c-3ffd-4a3f-b9c9-ba3d53db74d1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:51.641094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8c9bf-3024-47ff-9a23-6185d8fd8cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:51.641769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8c9bf-3024-47ff-9a23-6185d8fd8cef None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.642057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8c9bf-3024-47ff-9a23-6185d8fd8cef None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.642336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8c9bf-3024-47ff-9a23-6185d8fd8cef None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.643063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8c9bf-3024-47ff-9a23-6185d8fd8cef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:52:51.643459 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8c9bf-3024-47ff-9a23-6185d8fd8cef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:51.644899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8c9bf-3024-47ff-9a23-6185d8fd8cef None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:51.662525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 9/19] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:52:51.701545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:51.708918 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b5c7f-d9ac-4add-aaed-21b57b348990 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:51.754255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7ac17-bd03-4b45-8092-d23a8790cba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:51.755069 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7ac17-bd03-4b45-8092-d23a8790cba0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.755354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7ac17-bd03-4b45-8092-d23a8790cba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.755658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7ac17-bd03-4b45-8092-d23a8790cba0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.757774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7ac17-bd03-4b45-8092-d23a8790cba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:51.759481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7ac17-bd03-4b45-8092-d23a8790cba0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:51.761908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 11/20] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:52:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:52:51.767703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8c9bf-3024-47ff-9a23-6185d8fd8cef None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:51.804334 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:51.880569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8c9bf-3024-47ff-9a23-6185d8fd8cef None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:51.922627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b32ef-c9ac-4a17-9434-477cc62a5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:51.923611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b32ef-c9ac-4a17-9434-477cc62a5306 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.924045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b32ef-c9ac-4a17-9434-477cc62a5306 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.924467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b32ef-c9ac-4a17-9434-477cc62a5306 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.925486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b32ef-c9ac-4a17-9434-477cc62a5306 None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.domain.id=default, user.id=eae05ea06648449e9eda17d7c9a3f3fb) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:52:51.925801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b32ef-c9ac-4a17-9434-477cc62a5306 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:51.927324 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b32ef-c9ac-4a17-9434-477cc62a5306 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:51.935481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 10/21] 10.222.0.75 () {56 vars in 1322 bytes} [Thu Jul 14 10:52:51 2022] GET /identity/v3/role_assignments?user.id=eae05ea06648449e9eda17d7c9a3f3fb&scope.domain.id=default&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 233 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:52:54.253723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2ffb5-f414-425e-a934-74fe5531ea87 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:54.254323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2ffb5-f414-425e-a934-74fe5531ea87 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:54.254323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2ffb5-f414-425e-a934-74fe5531ea87 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:54.256078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 12/22] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:52:54.261548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345db058-3c0f-4602-bc81-e0062b7c43e0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:54.262402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345db058-3c0f-4602-bc81-e0062b7c43e0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:54.262902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345db058-3c0f-4602-bc81-e0062b7c43e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:54.283435 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-345db058-3c0f-4602-bc81-e0062b7c43e0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:54.284950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-345db058-3c0f-4602-bc81-e0062b7c43e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:54.375464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 11/23] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:52:54.409530 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c4015-4505-4892-9ea2-ab214a49ef63 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:54.410201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c4015-4505-4892-9ea2-ab214a49ef63 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:54.410789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c4015-4505-4892-9ea2-ab214a49ef63 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:54.413118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 13/24] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:52:54.414430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345db058-3c0f-4602-bc81-e0062b7c43e0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:54.428991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1f69b-756d-4059-8653-240b98be9b92 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:54.429625 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1f69b-756d-4059-8653-240b98be9b92 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:54.430123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1f69b-756d-4059-8653-240b98be9b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:54.457930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d4a1f69b-756d-4059-8653-240b98be9b92 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:54.459551 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d4a1f69b-756d-4059-8653-240b98be9b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:54.478203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:54.597779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 14/25] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:52:54.599239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2918b29-a07e-4fa7-85f5-702ac436eea9 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:54.599567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2918b29-a07e-4fa7-85f5-702ac436eea9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:54.600441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2918b29-a07e-4fa7-85f5-702ac436eea9 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:54.606910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345db058-3c0f-4602-bc81-e0062b7c43e0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:54.612059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 15/26] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:52:54 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:52:54.613516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38d48c-e46b-4dd6-bfaf-096e3ee841a0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:54.613816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38d48c-e46b-4dd6-bfaf-096e3ee841a0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:54.614105 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38d48c-e46b-4dd6-bfaf-096e3ee841a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:54.648432 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-bb38d48c-e46b-4dd6-bfaf-096e3ee841a0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:54.649662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bb38d48c-e46b-4dd6-bfaf-096e3ee841a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:54.669803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d71e8-a720-424d-8b1d-98838d4768be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:54.670624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d71e8-a720-424d-8b1d-98838d4768be None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:54.670907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d71e8-a720-424d-8b1d-98838d4768be None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:54.671186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d71e8-a720-424d-8b1d-98838d4768be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:54.691570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d71e8-a720-424d-8b1d-98838d4768be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:54.693202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d71e8-a720-424d-8b1d-98838d4768be None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f37d71e8-a720-424d-8b1d-98838d4768be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:52:54.728549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:52:54.729369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:52:54.730152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:52:54.730152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:52:54.730152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:52:54.730152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:52:54.732628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 12/27] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:52:54.733711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d71e8-a720-424d-8b1d-98838d4768be None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:54.753949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 16/28] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:52:54.755152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38d48c-e46b-4dd6-bfaf-096e3ee841a0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:54.799769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:54.890855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:54.930082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38d48c-e46b-4dd6-bfaf-096e3ee841a0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:54.932655 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d71e8-a720-424d-8b1d-98838d4768be None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:54.991454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe186952-d205-45ff-8880-f6aea829786f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:54.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe186952-d205-45ff-8880-f6aea829786f None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:54.993039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe186952-d205-45ff-8880-f6aea829786f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:54.993559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe186952-d205-45ff-8880-f6aea829786f None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:54.994795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe186952-d205-45ff-8880-f6aea829786f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:54.996242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ea1ba-8dd8-416a-9729-268abde90b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:54.996733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe186952-d205-45ff-8880-f6aea829786f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:54.997063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ea1ba-8dd8-416a-9729-268abde90b6b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:54.997666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ea1ba-8dd8-416a-9729-268abde90b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:54.998145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ea1ba-8dd8-416a-9729-268abde90b6b None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:54.999661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ea1ba-8dd8-416a-9729-268abde90b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:52:55.000396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ea1ba-8dd8-416a-9729-268abde90b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:55.002924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ea1ba-8dd8-416a-9729-268abde90b6b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:55.034117 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 17/29] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:52:55.040266 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ea1ba-8dd8-416a-9729-268abde90b6b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:55.124023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 13/30] 10.222.0.75 () {60 vars in 1131 bytes} [Thu Jul 14 10:52:54 2022] POST /identity/v3/roles => generated 219 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:52:55.126045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d387be4-b99a-4cbd-a3ec-cf2b1ad5a178 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:55.126542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d387be4-b99a-4cbd-a3ec-cf2b1ad5a178 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:55.127027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d387be4-b99a-4cbd-a3ec-cf2b1ad5a178 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:55.128736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 14/31] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:52:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:52:55.129850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d387be4-b99a-4cbd-a3ec-cf2b1ad5a178 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:55.150661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:55.185901 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ea1ba-8dd8-416a-9729-268abde90b6b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:55.256516 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d387be4-b99a-4cbd-a3ec-cf2b1ad5a178 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:55.259279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a403741-b060-4890-88c7-62ee5d93c4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:55.260067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a403741-b060-4890-88c7-62ee5d93c4d2 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:55.260623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a403741-b060-4890-88c7-62ee5d93c4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:55.261083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a403741-b060-4890-88c7-62ee5d93c4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:55.262013 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a403741-b060-4890-88c7-62ee5d93c4d2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:55.263686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a403741-b060-4890-88c7-62ee5d93c4d2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:55.300122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47e4e5-378d-44e4-844e-28d93cf31725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:55.301821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47e4e5-378d-44e4-844e-28d93cf31725 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:55.302317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47e4e5-378d-44e4-844e-28d93cf31725 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:55.302876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47e4e5-378d-44e4-844e-28d93cf31725 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:55.304423 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47e4e5-378d-44e4-844e-28d93cf31725 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:55.306345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47e4e5-378d-44e4-844e-28d93cf31725 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:55.318485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:55.344211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 15/32] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:52:55 2022] POST /identity/v3/projects => generated 301 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:52:55.345505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017195aa-3d4c-4711-ad0e-4ae63a3586e2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:55.345802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017195aa-3d4c-4711-ad0e-4ae63a3586e2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:55.346085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017195aa-3d4c-4711-ad0e-4ae63a3586e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:55.368339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-017195aa-3d4c-4711-ad0e-4ae63a3586e2 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:55.370001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-017195aa-3d4c-4711-ad0e-4ae63a3586e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:55.479551 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 18/33] 10.222.0.75 () {60 vars in 1131 bytes} [Thu Jul 14 10:52:55 2022] POST /identity/v3/roles => generated 225 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:52:55.481813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c8d07-8188-434b-91cb-58936936484e None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:55.482367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c8d07-8188-434b-91cb-58936936484e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:55.482864 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c8d07-8188-434b-91cb-58936936484e None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:55.484644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 19/34] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:52:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:52:55.493091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec4cc9-177a-485d-aa09-e71ca94ef014 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:55.493447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec4cc9-177a-485d-aa09-e71ca94ef014 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:55.493687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec4cc9-177a-485d-aa09-e71ca94ef014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:55.515067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3bec4cc9-177a-485d-aa09-e71ca94ef014 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:55.516748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3bec4cc9-177a-485d-aa09-e71ca94ef014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:55.583224 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 16/35] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:52:55.613715 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 20/36] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:52:55.626669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017195aa-3d4c-4711-ad0e-4ae63a3586e2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:55.663830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec4cc9-177a-485d-aa09-e71ca94ef014 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:55.750249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:55.774342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017195aa-3d4c-4711-ad0e-4ae63a3586e2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:55.786737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec4cc9-177a-485d-aa09-e71ca94ef014 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:55.805379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:55.818993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5812cd-3f29-42eb-9778-470d84176f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:55.819851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5812cd-3f29-42eb-9778-470d84176f78 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:55.820140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5812cd-3f29-42eb-9778-470d84176f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:55.820430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5812cd-3f29-42eb-9778-470d84176f78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:55.828766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908473ad-45d0-40a0-a6f8-80276eea9108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:55.829565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908473ad-45d0-40a0-a6f8-80276eea9108 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:55.829870 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908473ad-45d0-40a0-a6f8-80276eea9108 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:55.830153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908473ad-45d0-40a0-a6f8-80276eea9108 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:55.831566 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908473ad-45d0-40a0-a6f8-80276eea9108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:55.833369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908473ad-45d0-40a0-a6f8-80276eea9108 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:55.838074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 21/37] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:52:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:52:55.838206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5812cd-3f29-42eb-9778-470d84176f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:52:55.843782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908473ad-45d0-40a0-a6f8-80276eea9108 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-ea5812cd-3f29-42eb-9778-470d84176f78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:52:55.846150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:52:55.846955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:52:55.847745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 17/38] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:52:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:52:55.854783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5812cd-3f29-42eb-9778-470d84176f78 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:55.949464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908473ad-45d0-40a0-a6f8-80276eea9108 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:55.957286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5812cd-3f29-42eb-9778-470d84176f78 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:55.983516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbcb0c-a575-46fd-a431-b915bcc25fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:55.984186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbcb0c-a575-46fd-a431-b915bcc25fdc None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:55.984477 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbcb0c-a575-46fd-a431-b915bcc25fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:55.984765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbcb0c-a575-46fd-a431-b915bcc25fdc None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:55.985494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbcb0c-a575-46fd-a431-b915bcc25fdc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:55.986968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbcb0c-a575-46fd-a431-b915bcc25fdc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:55.997558 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f74b8a-7353-4aae-b19c-10abf2665088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:55.998341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f74b8a-7353-4aae-b19c-10abf2665088 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:55.998690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f74b8a-7353-4aae-b19c-10abf2665088 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:55.999078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f74b8a-7353-4aae-b19c-10abf2665088 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:56.000083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f74b8a-7353-4aae-b19c-10abf2665088 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:52:56.000457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f74b8a-7353-4aae-b19c-10abf2665088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:56.002160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f74b8a-7353-4aae-b19c-10abf2665088 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:56.002537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:56.017416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 18/39] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:52:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:52:56.018960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2b330-fb87-4935-bfcb-359d166e1f43 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:56.019299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2b330-fb87-4935-bfcb-359d166e1f43 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:56.019700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2b330-fb87-4935-bfcb-359d166e1f43 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:56.021438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 19/40] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:52:56.023575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2b330-fb87-4935-bfcb-359d166e1f43 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:56.035515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 22/41] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:52:55 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:52:56.037849 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc18ff-364b-4aa8-a1c7-e22060bb4326 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:56.038227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc18ff-364b-4aa8-a1c7-e22060bb4326 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:56.038632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc18ff-364b-4aa8-a1c7-e22060bb4326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:56.057735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9dcc18ff-364b-4aa8-a1c7-e22060bb4326 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:56.059244 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9dcc18ff-364b-4aa8-a1c7-e22060bb4326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:56.088336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:56.148298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 23/42] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:52:56.150261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2b330-fb87-4935-bfcb-359d166e1f43 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:56.171500 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc18ff-364b-4aa8-a1c7-e22060bb4326 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:56.197035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e4afb-5f1f-4865-8a31-792b4a3dda17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:56.197979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e4afb-5f1f-4865-8a31-792b4a3dda17 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:56.198277 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e4afb-5f1f-4865-8a31-792b4a3dda17 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:56.198565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e4afb-5f1f-4865-8a31-792b4a3dda17 None admin] PATH_INFO: `/v3/users/eae05ea06648449e9eda17d7c9a3f3fb` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:56.199892 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e4afb-5f1f-4865-8a31-792b4a3dda17 None admin] RBAC: Authorizing `identity:get_user(user_id=eae05ea06648449e9eda17d7c9a3f3fb)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:56.201433 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e4afb-5f1f-4865-8a31-792b4a3dda17 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:56.203177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 20/43] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:52:56 2022] GET /identity/v3/users/eae05ea06648449e9eda17d7c9a3f3fb => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:52:56.210831 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e4afb-5f1f-4865-8a31-792b4a3dda17 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:56.289332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc18ff-364b-4aa8-a1c7-e22060bb4326 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:56.316523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:56.332829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a350f-2439-4fa0-b1a6-1a3cb2cff451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:56.333519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a350f-2439-4fa0-b1a6-1a3cb2cff451 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:56.333813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a350f-2439-4fa0-b1a6-1a3cb2cff451 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:56.334147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a350f-2439-4fa0-b1a6-1a3cb2cff451 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:56.334860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e4afb-5f1f-4865-8a31-792b4a3dda17 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:56.335444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a350f-2439-4fa0-b1a6-1a3cb2cff451 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:56.336958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a350f-2439-4fa0-b1a6-1a3cb2cff451 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:56.351544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:56.355317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 24/44] 10.222.0.75 () {60 vars in 1131 bytes} [Thu Jul 14 10:52:56 2022] POST /identity/v3/roles => generated 223 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:52:56.379963 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85535d-cc52-423d-aed7-2c68ee630cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:56.380838 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85535d-cc52-423d-aed7-2c68ee630cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:56.381775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85535d-cc52-423d-aed7-2c68ee630cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:56.382018 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85535d-cc52-423d-aed7-2c68ee630cf3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:56.383468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85535d-cc52-423d-aed7-2c68ee630cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:56.385274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85535d-cc52-423d-aed7-2c68ee630cf3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:56.389598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 21/45] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:52:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:52:56.395834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a350f-2439-4fa0-b1a6-1a3cb2cff451 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:56.522399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a350f-2439-4fa0-b1a6-1a3cb2cff451 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:56.564653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241cb5b-0625-4e66-94a2-cbbd98b11fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:56.564869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241cb5b-0625-4e66-94a2-cbbd98b11fbd None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:56.565135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241cb5b-0625-4e66-94a2-cbbd98b11fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:56.565423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241cb5b-0625-4e66-94a2-cbbd98b11fbd None admin] PATH_INFO: `/v3/domains/default/users/eae05ea06648449e9eda17d7c9a3f3fb/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:56.567893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241cb5b-0625-4e66-94a2-cbbd98b11fbd None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=eae05ea06648449e9eda17d7c9a3f3fb, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:56.569930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241cb5b-0625-4e66-94a2-cbbd98b11fbd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:56.586700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 25/46] 10.222.0.75 () {58 vars in 1361 bytes} [Thu Jul 14 10:52:56 2022] PUT /identity/v3/domains/default/users/eae05ea06648449e9eda17d7c9a3f3fb/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:52:58.076589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77f7fb-8b6f-4288-bfe1-7e1e823ba7c2 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:58.077126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77f7fb-8b6f-4288-bfe1-7e1e823ba7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:58.077212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77f7fb-8b6f-4288-bfe1-7e1e823ba7c2 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:58.078524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 22/47] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:52:58.083858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dd900-680d-4a78-a473-6a4d9925d668 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:58.084780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dd900-680d-4a78-a473-6a4d9925d668 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:58.084943 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dd900-680d-4a78-a473-6a4d9925d668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:58.102260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-bf6dd900-680d-4a78-a473-6a4d9925d668 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:58.103602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bf6dd900-680d-4a78-a473-6a4d9925d668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:58.138508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:58.172876 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 26/48] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:52:58.196363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77f7fb-8b6f-4288-bfe1-7e1e823ba7c2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:58.299799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77f7fb-8b6f-4288-bfe1-7e1e823ba7c2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:58.333729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e5277-687c-457f-a725-757e6691aad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:58.335005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e5277-687c-457f-a725-757e6691aad4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:58.335297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e5277-687c-457f-a725-757e6691aad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:58.335609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e5277-687c-457f-a725-757e6691aad4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:58.336799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e5277-687c-457f-a725-757e6691aad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:58.338263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e5277-687c-457f-a725-757e6691aad4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:58.339937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 23/49] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:52:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:52:58.345878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dd900-680d-4a78-a473-6a4d9925d668 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:58.558675 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dd900-680d-4a78-a473-6a4d9925d668 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:58.592950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55901a3d-ac60-433c-9dc5-7815f056c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:58.593540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55901a3d-ac60-433c-9dc5-7815f056c283 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:58.593825 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55901a3d-ac60-433c-9dc5-7815f056c283 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:58.594107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55901a3d-ac60-433c-9dc5-7815f056c283 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:58.594821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55901a3d-ac60-433c-9dc5-7815f056c283 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:58.596300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55901a3d-ac60-433c-9dc5-7815f056c283 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:58.622331 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae546b-5e98-4796-b3a8-82bd71ca2f7b None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:58.622627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae546b-5e98-4796-b3a8-82bd71ca2f7b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:58.622785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 27/50] 10.222.0.75 () {60 vars in 1140 bytes} [Thu Jul 14 10:52:58 2022] POST /identity/v3/projects => generated 298 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:52:58.622921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae546b-5e98-4796-b3a8-82bd71ca2f7b None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:58.624426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 24/51] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:52:58.631857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fa9a0-b405-460d-937a-5898f50e7dbf None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:58.632166 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fa9a0-b405-460d-937a-5898f50e7dbf None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:58.632446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fa9a0-b405-460d-937a-5898f50e7dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:58.650605 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-395fa9a0-b405-460d-937a-5898f50e7dbf None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:58.651934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-395fa9a0-b405-460d-937a-5898f50e7dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:58.671806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:58.726659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 28/52] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:52:58.750173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae546b-5e98-4796-b3a8-82bd71ca2f7b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:58.805613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:58.861833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae546b-5e98-4796-b3a8-82bd71ca2f7b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:58.895910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4252ea0-5efa-4e5c-b0ae-a816d6f12613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:58.896734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4252ea0-5efa-4e5c-b0ae-a816d6f12613 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:58.897020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4252ea0-5efa-4e5c-b0ae-a816d6f12613 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:58.897296 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4252ea0-5efa-4e5c-b0ae-a816d6f12613 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:58.907624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4252ea0-5efa-4e5c-b0ae-a816d6f12613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-c4252ea0-5efa-4e5c-b0ae-a816d6f12613 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:52:58.915200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:52:58.916236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:52:58.917061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 25/53] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:52:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:52:58.923779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fa9a0-b405-460d-937a-5898f50e7dbf None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:59.043685 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fa9a0-b405-460d-937a-5898f50e7dbf None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:59.077790 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4471f0d-5150-4b51-8bef-2c24f98c4365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:59.078464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4471f0d-5150-4b51-8bef-2c24f98c4365 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:59.078751 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4471f0d-5150-4b51-8bef-2c24f98c4365 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:59.079033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4471f0d-5150-4b51-8bef-2c24f98c4365 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:59.079840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4471f0d-5150-4b51-8bef-2c24f98c4365 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:52:59.080155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4471f0d-5150-4b51-8bef-2c24f98c4365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:59.081544 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4471f0d-5150-4b51-8bef-2c24f98c4365 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:59.090033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 29/54] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:52:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:52:59.095527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4252ea0-5efa-4e5c-b0ae-a816d6f12613 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:59.200349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4252ea0-5efa-4e5c-b0ae-a816d6f12613 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:59.211319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:59.237488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9436ffe-0d9d-4e84-99b2-01da3c7ef284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:59.238368 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9436ffe-0d9d-4e84-99b2-01da3c7ef284 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:59.238698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9436ffe-0d9d-4e84-99b2-01da3c7ef284 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:59.238937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9436ffe-0d9d-4e84-99b2-01da3c7ef284 None admin] PATH_INFO: `/v3/users/eae05ea06648449e9eda17d7c9a3f3fb` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:59.241040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9436ffe-0d9d-4e84-99b2-01da3c7ef284 None admin] RBAC: Authorizing `identity:get_user(user_id=eae05ea06648449e9eda17d7c9a3f3fb)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:59.242520 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9436ffe-0d9d-4e84-99b2-01da3c7ef284 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:59.244210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 26/55] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:52:59 2022] GET /identity/v3/users/eae05ea06648449e9eda17d7c9a3f3fb => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:52:59.249833 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4471f0d-5150-4b51-8bef-2c24f98c4365 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:59.268566 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:59.352461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4471f0d-5150-4b51-8bef-2c24f98c4365 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:59.386029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5df847-f07d-4a1a-92aa-643a4af257d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:59.386832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5df847-f07d-4a1a-92aa-643a4af257d0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:59.387120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5df847-f07d-4a1a-92aa-643a4af257d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:59.387424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5df847-f07d-4a1a-92aa-643a4af257d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:59.388614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5df847-f07d-4a1a-92aa-643a4af257d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:59.390093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5df847-f07d-4a1a-92aa-643a4af257d0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:59.391750 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 30/56] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:52:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:52:59.398080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9436ffe-0d9d-4e84-99b2-01da3c7ef284 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:59.495665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9436ffe-0d9d-4e84-99b2-01da3c7ef284 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:59.528817 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ddb26-506a-4431-b2a9-760a255a0459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:59.529476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ddb26-506a-4431-b2a9-760a255a0459 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:59.529764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ddb26-506a-4431-b2a9-760a255a0459 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:59.530047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ddb26-506a-4431-b2a9-760a255a0459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:59.530879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ddb26-506a-4431-b2a9-760a255a0459 None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.domain.id=default, user.id=eae05ea06648449e9eda17d7c9a3f3fb) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:52:59.531201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ddb26-506a-4431-b2a9-760a255a0459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:59.532709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ddb26-506a-4431-b2a9-760a255a0459 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:59.539118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 27/57] 10.222.0.75 () {56 vars in 1322 bytes} [Thu Jul 14 10:52:59 2022] GET /identity/v3/role_assignments?user.id=eae05ea06648449e9eda17d7c9a3f3fb&scope.domain.id=default&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:00.691752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1f6a9-a744-4fd8-8a4e-df430e42fa4f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:00.692265 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1f6a9-a744-4fd8-8a4e-df430e42fa4f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:00.692375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1f6a9-a744-4fd8-8a4e-df430e42fa4f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:00.693709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 31/58] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:00.698856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be478ab4-0529-4539-960b-c0b4feb5346c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:00.699537 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be478ab4-0529-4539-960b-c0b4feb5346c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:00.699537 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be478ab4-0529-4539-960b-c0b4feb5346c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:00.716027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-be478ab4-0529-4539-960b-c0b4feb5346c None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:00.717222 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-be478ab4-0529-4539-960b-c0b4feb5346c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:00.752994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:00.798958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 28/59] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:00.824984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1f6a9-a744-4fd8-8a4e-df430e42fa4f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:00.920374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:00.944356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1f6a9-a744-4fd8-8a4e-df430e42fa4f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:00.980295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c677de1-f415-4f8f-953f-a2b59ad66ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:00.981114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c677de1-f415-4f8f-953f-a2b59ad66ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:00.981400 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c677de1-f415-4f8f-953f-a2b59ad66ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:00.981683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c677de1-f415-4f8f-953f-a2b59ad66ea4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:00.982866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c677de1-f415-4f8f-953f-a2b59ad66ea4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:00.984367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c677de1-f415-4f8f-953f-a2b59ad66ea4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:00.986001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 32/60] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:00.992218 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be478ab4-0529-4539-960b-c0b4feb5346c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:01.092130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be478ab4-0529-4539-960b-c0b4feb5346c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:01.129066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f2958-2ad9-4653-9033-aa9bcb26d076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:01.129804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f2958-2ad9-4653-9033-aa9bcb26d076 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:01.130088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f2958-2ad9-4653-9033-aa9bcb26d076 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:01.130391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f2958-2ad9-4653-9033-aa9bcb26d076 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:01.131765 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f2958-2ad9-4653-9033-aa9bcb26d076 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:01.133315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f2958-2ad9-4653-9033-aa9bcb26d076 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:01.157716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 29/61] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:53:00 2022] POST /identity/v3/users => generated 275 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:53:04.226988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87826c5c-383f-41ac-a4a2-418e807a3fa5 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:04.226988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87826c5c-383f-41ac-a4a2-418e807a3fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:04.226988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87826c5c-383f-41ac-a4a2-418e807a3fa5 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:04.226988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 33/62] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:04.233634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e2da2-14ad-4e1a-afdb-c76a221e8d94 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:04.234522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e2da2-14ad-4e1a-afdb-c76a221e8d94 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:04.235021 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e2da2-14ad-4e1a-afdb-c76a221e8d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:04.253851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8e9e2da2-14ad-4e1a-afdb-c76a221e8d94 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:04.255519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8e9e2da2-14ad-4e1a-afdb-c76a221e8d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:04.279579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:04.342938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 30/63] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:04.365577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87826c5c-383f-41ac-a4a2-418e807a3fa5 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:04.523249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87826c5c-383f-41ac-a4a2-418e807a3fa5 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:04.571469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf57af-3457-4ac7-943a-491372bcc47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:04.572290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf57af-3457-4ac7-943a-491372bcc47e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:04.572580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf57af-3457-4ac7-943a-491372bcc47e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:04.572861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf57af-3457-4ac7-943a-491372bcc47e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:04.595350 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf57af-3457-4ac7-943a-491372bcc47e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6ecf57af-3457-4ac7-943a-491372bcc47e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:04.603369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:04.604241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:04.604985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 34/64] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:04.610859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e2da2-14ad-4e1a-afdb-c76a221e8d94 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:04.673991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c6684-0218-4ecd-8e82-7d302f0b3cfe None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:04.674624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c6684-0218-4ecd-8e82-7d302f0b3cfe None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:04.675123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c6684-0218-4ecd-8e82-7d302f0b3cfe None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:04.676958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 35/65] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:04.683147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55ee9b-e26d-4f3d-92ce-cd8b53da2409 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:04.683729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55ee9b-e26d-4f3d-92ce-cd8b53da2409 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:04.684230 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55ee9b-e26d-4f3d-92ce-cd8b53da2409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:04.727007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9a55ee9b-e26d-4f3d-92ce-cd8b53da2409 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:04.728252 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9a55ee9b-e26d-4f3d-92ce-cd8b53da2409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:04.747142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e2da2-14ad-4e1a-afdb-c76a221e8d94 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:04.752642 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:04.793414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46992f1d-6cde-4002-a99c-d3c2a9288a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:04.794028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46992f1d-6cde-4002-a99c-d3c2a9288a40 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:04.794323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46992f1d-6cde-4002-a99c-d3c2a9288a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:04.794609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46992f1d-6cde-4002-a99c-d3c2a9288a40 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:04.795395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46992f1d-6cde-4002-a99c-d3c2a9288a40 None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:04.795713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46992f1d-6cde-4002-a99c-d3c2a9288a40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:04.797135 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46992f1d-6cde-4002-a99c-d3c2a9288a40 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:04.815105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 31/66] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:04.821373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46992f1d-6cde-4002-a99c-d3c2a9288a40 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:04.848613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:04.868239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 36/67] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:04.870150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b46e3-3805-4192-b256-918fce4ec4a3 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:04.871600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b46e3-3805-4192-b256-918fce4ec4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:04.872075 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b46e3-3805-4192-b256-918fce4ec4a3 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:04.873887 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 37/68] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:04.880826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83a2c2-7539-4ffb-8e67-fa9afbb1e20c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:04.881120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83a2c2-7539-4ffb-8e67-fa9afbb1e20c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:04.881979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83a2c2-7539-4ffb-8e67-fa9afbb1e20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:04.909077 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2b83a2c2-7539-4ffb-8e67-fa9afbb1e20c None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:04.910543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2b83a2c2-7539-4ffb-8e67-fa9afbb1e20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:04.976474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46992f1d-6cde-4002-a99c-d3c2a9288a40 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:04.992137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 38/69] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:04.993214 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83a2c2-7539-4ffb-8e67-fa9afbb1e20c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:05.036366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1fac8-ac10-473d-88c9-57791d3216a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:05.038152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1fac8-ac10-473d-88c9-57791d3216a6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.038633 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1fac8-ac10-473d-88c9-57791d3216a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.039124 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1fac8-ac10-473d-88c9-57791d3216a6 None admin] PATH_INFO: `/v3/users/5e6846265bb648f1b7034eadef52b2d8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.054135 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1fac8-ac10-473d-88c9-57791d3216a6 None admin] RBAC: Authorizing `identity:get_user(user_id=5e6846265bb648f1b7034eadef52b2d8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:05.055583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1fac8-ac10-473d-88c9-57791d3216a6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:05.059565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 32/70] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:04 2022] GET /identity/v3/users/5e6846265bb648f1b7034eadef52b2d8 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:05.059565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1fac8-ac10-473d-88c9-57791d3216a6 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:05.075373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:05.111050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83a2c2-7539-4ffb-8e67-fa9afbb1e20c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:05.134259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:05.145856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f4cb1-50a9-4d81-aed5-5b571a876bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:05.146667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f4cb1-50a9-4d81-aed5-5b571a876bff None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.146983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f4cb1-50a9-4d81-aed5-5b571a876bff None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.147275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f4cb1-50a9-4d81-aed5-5b571a876bff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.157594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1fac8-ac10-473d-88c9-57791d3216a6 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:05.157729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f4cb1-50a9-4d81-aed5-5b571a876bff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-322f4cb1-50a9-4d81-aed5-5b571a876bff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:05.166043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:05.166937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:05.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 39/71] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:53:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:05.168523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de726313-cac4-4c7d-a989-69f6965be5d8 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.168805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de726313-cac4-4c7d-a989-69f6965be5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.169114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de726313-cac4-4c7d-a989-69f6965be5d8 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.170364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 40/72] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:05.171081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de726313-cac4-4c7d-a989-69f6965be5d8 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:05.208038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8fefb-c861-4d5b-b815-5e7fce8046f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:05.208862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8fefb-c861-4d5b-b815-5e7fce8046f4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.209149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8fefb-c861-4d5b-b815-5e7fce8046f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.209436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8fefb-c861-4d5b-b815-5e7fce8046f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.219907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8fefb-c861-4d5b-b815-5e7fce8046f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-2ce8fefb-c861-4d5b-b815-5e7fce8046f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:05.228636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:05.229707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:05.230622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 33/73] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:05.231417 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92ed8-a3b3-4d34-88e7-ccffa0883d4d None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.231507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92ed8-a3b3-4d34-88e7-ccffa0883d4d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.231748 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92ed8-a3b3-4d34-88e7-ccffa0883d4d None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.235420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 34/74] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:05.235420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92ed8-a3b3-4d34-88e7-ccffa0883d4d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:05.287212 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de726313-cac4-4c7d-a989-69f6965be5d8 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:05.333507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:05.339891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31466e-33c8-484f-84c1-85b54541496e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:05.340553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31466e-33c8-484f-84c1-85b54541496e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.340837 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31466e-33c8-484f-84c1-85b54541496e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.341163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31466e-33c8-484f-84c1-85b54541496e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.349797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92ed8-a3b3-4d34-88e7-ccffa0883d4d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:05.387670 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5bf98-1bc2-4621-af24-ed959119fc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:05.388350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5bf98-1bc2-4621-af24-ed959119fc1d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.388641 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5bf98-1bc2-4621-af24-ed959119fc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.388938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5bf98-1bc2-4621-af24-ed959119fc1d None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.389695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5bf98-1bc2-4621-af24-ed959119fc1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:05.390008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5bf98-1bc2-4621-af24-ed959119fc1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:05.391439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5bf98-1bc2-4621-af24-ed959119fc1d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:05.393848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:05.401196 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 35/75] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:05.402412 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd218b7-28ce-44d8-8303-43b6bc6b830c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.402700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd218b7-28ce-44d8-8303-43b6bc6b830c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.402981 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd218b7-28ce-44d8-8303-43b6bc6b830c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.413157 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31466e-33c8-484f-84c1-85b54541496e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:05.414658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31466e-33c8-484f-84c1-85b54541496e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:05.419741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6fd218b7-28ce-44d8-8303-43b6bc6b830c None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:05.420946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6fd218b7-28ce-44d8-8303-43b6bc6b830c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:05.491807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 36/76] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:05.492679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd218b7-28ce-44d8-8303-43b6bc6b830c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:05.519819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 41/77] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:05.521512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba330e33-1d50-4804-bb70-679b25316dcc None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.521831 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba330e33-1d50-4804-bb70-679b25316dcc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.522664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba330e33-1d50-4804-bb70-679b25316dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.539588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ba330e33-1d50-4804-bb70-679b25316dcc None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:05.540813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ba330e33-1d50-4804-bb70-679b25316dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:05.567091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:05.613028 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 42/78] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:05.613546 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd218b7-28ce-44d8-8303-43b6bc6b830c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:05.613802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba330e33-1d50-4804-bb70-679b25316dcc None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:05.651446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:05.659307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570c828-bc5b-4b5f-a29b-1eaf676eebbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:05.660019 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570c828-bc5b-4b5f-a29b-1eaf676eebbe None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.660304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570c828-bc5b-4b5f-a29b-1eaf676eebbe None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.660585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570c828-bc5b-4b5f-a29b-1eaf676eebbe None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.661341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570c828-bc5b-4b5f-a29b-1eaf676eebbe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:05.661651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570c828-bc5b-4b5f-a29b-1eaf676eebbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:05.663640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570c828-bc5b-4b5f-a29b-1eaf676eebbe None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:05.675578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 37/79] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:05.676424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570c828-bc5b-4b5f-a29b-1eaf676eebbe None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:05.724100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba330e33-1d50-4804-bb70-679b25316dcc None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:05.759126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938b660-2a54-460c-b8d4-5c0acafb1ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:05.760046 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938b660-2a54-460c-b8d4-5c0acafb1ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.760332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938b660-2a54-460c-b8d4-5c0acafb1ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.760609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938b660-2a54-460c-b8d4-5c0acafb1ee2 None admin] PATH_INFO: `/v3/users/eae05ea06648449e9eda17d7c9a3f3fb` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.761817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938b660-2a54-460c-b8d4-5c0acafb1ee2 None admin] RBAC: Authorizing `identity:get_user(user_id=eae05ea06648449e9eda17d7c9a3f3fb)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:05.763280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938b660-2a54-460c-b8d4-5c0acafb1ee2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:05.764951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 43/80] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:05 2022] GET /identity/v3/users/eae05ea06648449e9eda17d7c9a3f3fb => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:05.765714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938b660-2a54-460c-b8d4-5c0acafb1ee2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:05.780971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570c828-bc5b-4b5f-a29b-1eaf676eebbe None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:05.796557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:05.815039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fa7f3-1a4b-42c1-b40f-9a5111da4b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:05.815886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fa7f3-1a4b-42c1-b40f-9a5111da4b3f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.816175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fa7f3-1a4b-42c1-b40f-9a5111da4b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.816455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fa7f3-1a4b-42c1-b40f-9a5111da4b3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.827198 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fa7f3-1a4b-42c1-b40f-9a5111da4b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-e76fa7f3-1a4b-42c1-b40f-9a5111da4b3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:05.835404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:05.837073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 38/81] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:05.838606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fa7f3-1a4b-42c1-b40f-9a5111da4b3f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:05.880507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938b660-2a54-460c-b8d4-5c0acafb1ee2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:05.902570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:05.919840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd6b80-c4f1-4975-9038-58824fd2aa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:05.920882 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd6b80-c4f1-4975-9038-58824fd2aa36 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.921242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd6b80-c4f1-4975-9038-58824fd2aa36 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.921620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd6b80-c4f1-4975-9038-58824fd2aa36 None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.923005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd6b80-c4f1-4975-9038-58824fd2aa36 None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:05.924743 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd6b80-c4f1-4975-9038-58824fd2aa36 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:05.926866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 44/82] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:05 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:05.927795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd6b80-c4f1-4975-9038-58824fd2aa36 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:05.955759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fa7f3-1a4b-42c1-b40f-9a5111da4b3f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:05.993285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da382f5-0a1a-4a7e-a96e-88fddc2b0a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:05.994335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da382f5-0a1a-4a7e-a96e-88fddc2b0a7f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:05.994773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da382f5-0a1a-4a7e-a96e-88fddc2b0a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:05.995061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da382f5-0a1a-4a7e-a96e-88fddc2b0a7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:05.996294 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da382f5-0a1a-4a7e-a96e-88fddc2b0a7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:05.997765 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da382f5-0a1a-4a7e-a96e-88fddc2b0a7f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:05.999404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 39/83] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:06.000167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da382f5-0a1a-4a7e-a96e-88fddc2b0a7f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:06.035612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:06.040303 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd6b80-c4f1-4975-9038-58824fd2aa36 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:06.075006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643329b-3ce8-41ca-9ebe-b5f29cb5c553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:06.075939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643329b-3ce8-41ca-9ebe-b5f29cb5c553 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:06.076233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643329b-3ce8-41ca-9ebe-b5f29cb5c553 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:06.076516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643329b-3ce8-41ca-9ebe-b5f29cb5c553 None admin] PATH_INFO: `/v3/users/5e6846265bb648f1b7034eadef52b2d8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:06.077733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643329b-3ce8-41ca-9ebe-b5f29cb5c553 None admin] RBAC: Authorizing `identity:get_user(user_id=5e6846265bb648f1b7034eadef52b2d8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:06.079218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643329b-3ce8-41ca-9ebe-b5f29cb5c553 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:06.081080 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 45/84] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:05 2022] GET /identity/v3/users/5e6846265bb648f1b7034eadef52b2d8 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:06.081672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643329b-3ce8-41ca-9ebe-b5f29cb5c553 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:06.100896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da382f5-0a1a-4a7e-a96e-88fddc2b0a7f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:06.118050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:06.135827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9e148-2c1b-4e39-a95f-82b10f935065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:06.136702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9e148-2c1b-4e39-a95f-82b10f935065 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:06.136991 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9e148-2c1b-4e39-a95f-82b10f935065 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:06.137275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9e148-2c1b-4e39-a95f-82b10f935065 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:06.185758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643329b-3ce8-41ca-9ebe-b5f29cb5c553 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:06.205040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9e148-2c1b-4e39-a95f-82b10f935065 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:06.206562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9e148-2c1b-4e39-a95f-82b10f935065 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:06.223315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a243ff5-728a-4f49-bc92-3f298f6868c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:06.224023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a243ff5-728a-4f49-bc92-3f298f6868c3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:06.224313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a243ff5-728a-4f49-bc92-3f298f6868c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:06.224597 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a243ff5-728a-4f49-bc92-3f298f6868c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:06.225379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a243ff5-728a-4f49-bc92-3f298f6868c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:06.225692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a243ff5-728a-4f49-bc92-3f298f6868c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:06.227070 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a243ff5-728a-4f49-bc92-3f298f6868c3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:06.235674 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 46/85] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:06.236260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a243ff5-728a-4f49-bc92-3f298f6868c3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:06.265166 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:06.319513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 40/86] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:06.320528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9e148-2c1b-4e39-a95f-82b10f935065 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:06.339070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:06.341742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a243ff5-728a-4f49-bc92-3f298f6868c3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:06.376020 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7180f5-b3bd-4ae7-81b8-b06eafc188fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:06.376696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7180f5-b3bd-4ae7-81b8-b06eafc188fc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:06.376988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7180f5-b3bd-4ae7-81b8-b06eafc188fc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:06.377280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7180f5-b3bd-4ae7-81b8-b06eafc188fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:06.378120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7180f5-b3bd-4ae7-81b8-b06eafc188fc None admin] RBAC: Adding query filter params (role.id=9a562f9a0cad4304852f3183eef398d2, scope.project.id=cc744332e3b04129956875355de3f205, user.id=5e6846265bb648f1b7034eadef52b2d8) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:06.378444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7180f5-b3bd-4ae7-81b8-b06eafc188fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:06.379960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7180f5-b3bd-4ae7-81b8-b06eafc188fc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:06.388348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 47/87] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:06 2022] GET /identity/v3/role_assignments?user.id=5e6846265bb648f1b7034eadef52b2d8&scope.project.id=cc744332e3b04129956875355de3f205&role.id=9a562f9a0cad4304852f3183eef398d2 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:06.389104 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7180f5-b3bd-4ae7-81b8-b06eafc188fc None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:06.424232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9e148-2c1b-4e39-a95f-82b10f935065 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:06.464361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257ad17-3fb5-4dcd-9a89-158fd0e4e967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:06.465063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257ad17-3fb5-4dcd-9a89-158fd0e4e967 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:06.465359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257ad17-3fb5-4dcd-9a89-158fd0e4e967 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:06.465646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257ad17-3fb5-4dcd-9a89-158fd0e4e967 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:06.466395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257ad17-3fb5-4dcd-9a89-158fd0e4e967 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:06.467961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257ad17-3fb5-4dcd-9a89-158fd0e4e967 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:06.491908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 41/88] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:53:06 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:53:06.504496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257ad17-3fb5-4dcd-9a89-158fd0e4e967 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:06.511927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:06.523565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7180f5-b3bd-4ae7-81b8-b06eafc188fc None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:06.570351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a18b94-279d-469f-bba4-e261f1a650a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:06.571041 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18b94-279d-469f-bba4-e261f1a650a7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:06.571335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18b94-279d-469f-bba4-e261f1a650a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:06.571687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18b94-279d-469f-bba4-e261f1a650a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:06.635922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:06.648172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257ad17-3fb5-4dcd-9a89-158fd0e4e967 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:06.666339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a18b94-279d-469f-bba4-e261f1a650a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:06.667066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a18b94-279d-469f-bba4-e261f1a650a7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:06.693577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb48f-983c-4cc5-8858-eafb5e4d93ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:06.694839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb48f-983c-4cc5-8858-eafb5e4d93ae None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:06.695133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb48f-983c-4cc5-8858-eafb5e4d93ae None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:06.695462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb48f-983c-4cc5-8858-eafb5e4d93ae None admin] PATH_INFO: `/v3/users/5e6846265bb648f1b7034eadef52b2d8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:06.696737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cb48f-983c-4cc5-8858-eafb5e4d93ae None admin] RBAC: Authorizing `identity:get_user(user_id=5e6846265bb648f1b7034eadef52b2d8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:06.698245 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cb48f-983c-4cc5-8858-eafb5e4d93ae None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:06.699965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 42/89] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:06 2022] GET /identity/v3/users/5e6846265bb648f1b7034eadef52b2d8 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:06.700753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb48f-983c-4cc5-8858-eafb5e4d93ae None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:06.772491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:06.785224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 48/90] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:06.785914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a18b94-279d-469f-bba4-e261f1a650a7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:06.827291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb48f-983c-4cc5-8858-eafb5e4d93ae None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:06.874306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080155f-8702-4383-a9d8-464962f2beda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:06.875645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080155f-8702-4383-a9d8-464962f2beda None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:06.876624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080155f-8702-4383-a9d8-464962f2beda None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:06.876624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080155f-8702-4383-a9d8-464962f2beda None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:06.877593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080155f-8702-4383-a9d8-464962f2beda None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:06.879172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080155f-8702-4383-a9d8-464962f2beda None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:06.881072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 43/91] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:06 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:06.882326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080155f-8702-4383-a9d8-464962f2beda None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:06.902815 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:06.946054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a18b94-279d-469f-bba4-e261f1a650a7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:07.007879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457f484-e1cd-436c-9e3c-077fbf958f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:07.008587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457f484-e1cd-436c-9e3c-077fbf958f13 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:07.009078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457f484-e1cd-436c-9e3c-077fbf958f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:07.009564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457f484-e1cd-436c-9e3c-077fbf958f13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:07.083572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080155f-8702-4383-a9d8-464962f2beda None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:07.132991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:07.151480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457f484-e1cd-436c-9e3c-077fbf958f13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:07.152566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4995ced-0e80-4a5a-a033-c4d521692e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:07.154117 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457f484-e1cd-436c-9e3c-077fbf958f13 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:07.154582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4995ced-0e80-4a5a-a033-c4d521692e9c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:07.155288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4995ced-0e80-4a5a-a033-c4d521692e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:07.156085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4995ced-0e80-4a5a-a033-c4d521692e9c None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:07.160105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4995ced-0e80-4a5a-a033-c4d521692e9c None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:07.163746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4995ced-0e80-4a5a-a033-c4d521692e9c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:07.167650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 44/92] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:06 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:07.169309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4995ced-0e80-4a5a-a033-c4d521692e9c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:07.328670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 49/93] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:07.329770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457f484-e1cd-436c-9e3c-077fbf958f13 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:07.383113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:07.438261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4995ced-0e80-4a5a-a033-c4d521692e9c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:07.438427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:07.449469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457f484-e1cd-436c-9e3c-077fbf958f13 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:07.508098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e35f2-4bd7-4174-badf-8579e85c140b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:07.508771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e35f2-4bd7-4174-badf-8579e85c140b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:07.509064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e35f2-4bd7-4174-badf-8579e85c140b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:07.509353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e35f2-4bd7-4174-badf-8579e85c140b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:07.510202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e35f2-4bd7-4174-badf-8579e85c140b None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=cc744332e3b04129956875355de3f205, user.id=eae05ea06648449e9eda17d7c9a3f3fb) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:07.510524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e35f2-4bd7-4174-badf-8579e85c140b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:07.512023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e35f2-4bd7-4174-badf-8579e85c140b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:07.518851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7eb28b-a78a-4748-bb43-6d41eb402fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:07.519555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7eb28b-a78a-4748-bb43-6d41eb402fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:07.519848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7eb28b-a78a-4748-bb43-6d41eb402fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:07.520131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7eb28b-a78a-4748-bb43-6d41eb402fa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:07.521252 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7eb28b-a78a-4748-bb43-6d41eb402fa5 None admin] RBAC: Adding query filter params (role.id=ff3de46e4867421f990563b41a305d4d, scope.project.id=cc744332e3b04129956875355de3f205, user.id=5e6846265bb648f1b7034eadef52b2d8) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:07.521575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7eb28b-a78a-4748-bb43-6d41eb402fa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:07.526855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 45/94] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:07 2022] GET /identity/v3/role_assignments?user.id=eae05ea06648449e9eda17d7c9a3f3fb&scope.project.id=cc744332e3b04129956875355de3f205&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 259 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:07.527520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7eb28b-a78a-4748-bb43-6d41eb402fa5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:07.530295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e35f2-4bd7-4174-badf-8579e85c140b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:07.538449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 50/95] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:07 2022] GET /identity/v3/role_assignments?user.id=5e6846265bb648f1b7034eadef52b2d8&scope.project.id=cc744332e3b04129956875355de3f205&role.id=ff3de46e4867421f990563b41a305d4d => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:07.663375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e35f2-4bd7-4174-badf-8579e85c140b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:07.714933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0abdd-9063-4766-b9f5-aa9def1eb622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:07.716191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0abdd-9063-4766-b9f5-aa9def1eb622 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:07.716710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0abdd-9063-4766-b9f5-aa9def1eb622 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:07.717193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0abdd-9063-4766-b9f5-aa9def1eb622 None admin] PATH_INFO: `/v3/projects/d152cbf8b76149db89069b619ae0f5c1` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:07.718749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0abdd-9063-4766-b9f5-aa9def1eb622 None admin] RBAC: Authorizing `identity:get_project(project_id=d152cbf8b76149db89069b619ae0f5c1)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:07.720485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0abdd-9063-4766-b9f5-aa9def1eb622 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:07.722468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 46/96] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:07 2022] GET /identity/v3/projects/d152cbf8b76149db89069b619ae0f5c1 => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:07.729279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7eb28b-a78a-4748-bb43-6d41eb402fa5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:07.883421 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7eb28b-a78a-4748-bb43-6d41eb402fa5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:07.930823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50928548-0090-4340-a85a-370329277c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:07.931518 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50928548-0090-4340-a85a-370329277c1f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:07.931814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50928548-0090-4340-a85a-370329277c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:07.932101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50928548-0090-4340-a85a-370329277c1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:07.939679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50928548-0090-4340-a85a-370329277c1f None admin] RBAC: Adding query filter params (role.id=9a562f9a0cad4304852f3183eef398d2, scope.project.id=d152cbf8b76149db89069b619ae0f5c1, user.id=5e6846265bb648f1b7034eadef52b2d8) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:07.940039 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50928548-0090-4340-a85a-370329277c1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:07.941513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50928548-0090-4340-a85a-370329277c1f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:07.948053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 51/97] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:07 2022] GET /identity/v3/role_assignments?user.id=5e6846265bb648f1b7034eadef52b2d8&scope.project.id=d152cbf8b76149db89069b619ae0f5c1&role.id=9a562f9a0cad4304852f3183eef398d2 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:08.863014 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933adac-4962-4237-92b8-cb63d9210257 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:08.863554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933adac-4962-4237-92b8-cb63d9210257 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:08.863627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933adac-4962-4237-92b8-cb63d9210257 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:08.864972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 47/98] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:08.870766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565fd8-2881-408d-a0eb-d2487fdd04b0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:08.871190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565fd8-2881-408d-a0eb-d2487fdd04b0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:08.871402 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565fd8-2881-408d-a0eb-d2487fdd04b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:08.888205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f4565fd8-2881-408d-a0eb-d2487fdd04b0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:08.889415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f4565fd8-2881-408d-a0eb-d2487fdd04b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:08.903866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:08.973633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 52/99] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:09.012089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933adac-4962-4237-92b8-cb63d9210257 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:09.026153 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:09.151014 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933adac-4962-4237-92b8-cb63d9210257 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:09.206505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d9766-a79f-4350-b077-fb2b2e41a612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:09.207322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d9766-a79f-4350-b077-fb2b2e41a612 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:09.207637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d9766-a79f-4350-b077-fb2b2e41a612 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:09.207921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d9766-a79f-4350-b077-fb2b2e41a612 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:09.227911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d9766-a79f-4350-b077-fb2b2e41a612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-826d9766-a79f-4350-b077-fb2b2e41a612 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:09.235627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:09.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:09.237212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 48/100] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:09.243747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565fd8-2881-408d-a0eb-d2487fdd04b0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:09.383355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565fd8-2881-408d-a0eb-d2487fdd04b0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:09.445328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59577bb-b755-43a3-8062-5ba9dc3bef9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:09.445956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59577bb-b755-43a3-8062-5ba9dc3bef9f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:09.446246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59577bb-b755-43a3-8062-5ba9dc3bef9f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:09.446529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59577bb-b755-43a3-8062-5ba9dc3bef9f None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:09.447298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59577bb-b755-43a3-8062-5ba9dc3bef9f None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:09.447637 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59577bb-b755-43a3-8062-5ba9dc3bef9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:09.449046 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59577bb-b755-43a3-8062-5ba9dc3bef9f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:09.458375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 53/101] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:09.464365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d9766-a79f-4350-b077-fb2b2e41a612 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:09.542879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:09.579039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d9766-a79f-4350-b077-fb2b2e41a612 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:09.622360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31de93-d8e8-4835-a2cd-33712cb39d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:09.623238 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31de93-d8e8-4835-a2cd-33712cb39d64 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:09.623548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31de93-d8e8-4835-a2cd-33712cb39d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:09.623834 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31de93-d8e8-4835-a2cd-33712cb39d64 None admin] PATH_INFO: `/v3/users/5e6846265bb648f1b7034eadef52b2d8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:09.625037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31de93-d8e8-4835-a2cd-33712cb39d64 None admin] RBAC: Authorizing `identity:get_user(user_id=5e6846265bb648f1b7034eadef52b2d8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:09.627327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31de93-d8e8-4835-a2cd-33712cb39d64 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:09.629979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 49/102] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:09 2022] GET /identity/v3/users/5e6846265bb648f1b7034eadef52b2d8 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:09.636616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59577bb-b755-43a3-8062-5ba9dc3bef9f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:09.657601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:09.831995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59577bb-b755-43a3-8062-5ba9dc3bef9f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:09.849151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b5913-b311-4051-9f5f-5480e1250a50 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:09.849458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b5913-b311-4051-9f5f-5480e1250a50 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:09.849719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b5913-b311-4051-9f5f-5480e1250a50 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:09.851310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 50/103] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:09.868448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba12db-023b-477c-8efe-cacf6a9fe561 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:09.869045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba12db-023b-477c-8efe-cacf6a9fe561 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:09.869045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba12db-023b-477c-8efe-cacf6a9fe561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:09.881459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0795f9-a809-4fb7-8d01-4a3a7fb9389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:09.883245 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0795f9-a809-4fb7-8d01-4a3a7fb9389a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:09.883579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0795f9-a809-4fb7-8d01-4a3a7fb9389a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:09.883866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0795f9-a809-4fb7-8d01-4a3a7fb9389a None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:09.888141 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0795f9-a809-4fb7-8d01-4a3a7fb9389a None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:09.889658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0795f9-a809-4fb7-8d01-4a3a7fb9389a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:09.891975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 54/104] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:09 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:09.901598 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0795f9-a809-4fb7-8d01-4a3a7fb9389a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:09.904415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-98ba12db-023b-477c-8efe-cacf6a9fe561 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:09.905634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-98ba12db-023b-477c-8efe-cacf6a9fe561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:10.000000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 51/105] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:10.001389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d85fe8-1112-4681-9532-b52551153ca6 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:10.001721 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d85fe8-1112-4681-9532-b52551153ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:10.002012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d85fe8-1112-4681-9532-b52551153ca6 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:10.003807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 52/106] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:10.010491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea5076-23ed-4656-8843-561422dd6450 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:10.010804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea5076-23ed-4656-8843-561422dd6450 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:10.011092 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea5076-23ed-4656-8843-561422dd6450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:10.038967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-93ea5076-23ed-4656-8843-561422dd6450 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:10.040954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-93ea5076-23ed-4656-8843-561422dd6450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:10.102915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:10.123770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:10.133602 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 53/107] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:10.134566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea5076-23ed-4656-8843-561422dd6450 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:10.160702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0795f9-a809-4fb7-8d01-4a3a7fb9389a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:10.248534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea5076-23ed-4656-8843-561422dd6450 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:10.282973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5baa3eb-5d1b-4873-a0a8-0fb16460a47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:10.284219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5baa3eb-5d1b-4873-a0a8-0fb16460a47b None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:10.284508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5baa3eb-5d1b-4873-a0a8-0fb16460a47b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:10.284847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5baa3eb-5d1b-4873-a0a8-0fb16460a47b None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205/users/5e6846265bb648f1b7034eadef52b2d8/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:10.285118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a541ea-d16b-4566-a8ad-c2783d861b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:10.285930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a541ea-d16b-4566-a8ad-c2783d861b07 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:10.286220 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a541ea-d16b-4566-a8ad-c2783d861b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:10.286501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a541ea-d16b-4566-a8ad-c2783d861b07 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:10.286915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5baa3eb-5d1b-4873-a0a8-0fb16460a47b None admin] RBAC: Authorizing `identity:create_grant(project_id=cc744332e3b04129956875355de3f205, user_id=5e6846265bb648f1b7034eadef52b2d8, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:10.288935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5baa3eb-5d1b-4873-a0a8-0fb16460a47b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:10.308289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a541ea-d16b-4566-a8ad-c2783d861b07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:10.315789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 55/108] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:53:09 2022] PUT /identity/v3/projects/cc744332e3b04129956875355de3f205/users/5e6846265bb648f1b7034eadef52b2d8/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:53:10.316453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5baa3eb-5d1b-4873-a0a8-0fb16460a47b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-92a541ea-d16b-4566-a8ad-c2783d861b07 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:10.344249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:10.345176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:10.345909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:10.345909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:10.345909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:10.345909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:10.345909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:10.345909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:10.345909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:10.345909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:10.345909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:10.345909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:10.345909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:10.345909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:10.351412 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 54/109] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:10.354749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a541ea-d16b-4566-a8ad-c2783d861b07 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:10.508747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5baa3eb-5d1b-4873-a0a8-0fb16460a47b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:10.548257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d697cf-0aa7-4818-bd0c-9a4de7248c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:10.549090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d697cf-0aa7-4818-bd0c-9a4de7248c6b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:10.549367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d697cf-0aa7-4818-bd0c-9a4de7248c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:10.549647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d697cf-0aa7-4818-bd0c-9a4de7248c6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:10.550912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d697cf-0aa7-4818-bd0c-9a4de7248c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:10.556729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d697cf-0aa7-4818-bd0c-9a4de7248c6b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:10.558914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 56/110] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:10.563843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:10.564710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d697cf-0aa7-4818-bd0c-9a4de7248c6b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:10.583759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:10.658940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a541ea-d16b-4566-a8ad-c2783d861b07 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:10.696706 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d697cf-0aa7-4818-bd0c-9a4de7248c6b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:10.701707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ab8fc-7a04-47ea-87ee-a42eff559065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:10.702399 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ab8fc-7a04-47ea-87ee-a42eff559065 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:10.702693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ab8fc-7a04-47ea-87ee-a42eff559065 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:10.702981 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ab8fc-7a04-47ea-87ee-a42eff559065 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:10.703793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ab8fc-7a04-47ea-87ee-a42eff559065 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:10.704109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ab8fc-7a04-47ea-87ee-a42eff559065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:10.705586 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ab8fc-7a04-47ea-87ee-a42eff559065 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:10.715013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 55/111] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:10.716512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537eefa-3400-45bf-83c9-a5e133c13d39 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:10.716812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537eefa-3400-45bf-83c9-a5e133c13d39 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:10.717094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537eefa-3400-45bf-83c9-a5e133c13d39 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:10.718646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 56/112] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:10.720181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01cddb-d476-42a9-b043-01307fcfb1bb None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:10.720479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01cddb-d476-42a9-b043-01307fcfb1bb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:10.724550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01cddb-d476-42a9-b043-01307fcfb1bb None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:10.726025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 57/113] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:10.726926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01cddb-d476-42a9-b043-01307fcfb1bb None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:10.744325 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad933cd-f126-4cba-a768-dd8b48d6d237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:10.745293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad933cd-f126-4cba-a768-dd8b48d6d237 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:10.745777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad933cd-f126-4cba-a768-dd8b48d6d237 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:10.746338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad933cd-f126-4cba-a768-dd8b48d6d237 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:10.747351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad933cd-f126-4cba-a768-dd8b48d6d237 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:10.748997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad933cd-f126-4cba-a768-dd8b48d6d237 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:10.780553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 57/114] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:53:10 2022] POST /identity/v3/projects => generated 302 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:53:10.782023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c58cf-7c54-40e7-b95e-4e0a06228ed6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:10.782328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c58cf-7c54-40e7-b95e-4e0a06228ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:10.782614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c58cf-7c54-40e7-b95e-4e0a06228ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:10.808372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9d0c58cf-7c54-40e7-b95e-4e0a06228ed6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:10.810059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9d0c58cf-7c54-40e7-b95e-4e0a06228ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:10.881404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01cddb-d476-42a9-b043-01307fcfb1bb None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:10.888178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:10.916893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:10.923779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 58/115] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:10.927141 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cdacd-3a2e-45e1-95e7-314d99ebfb14 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:10.927723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cdacd-3a2e-45e1-95e7-314d99ebfb14 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:10.928279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cdacd-3a2e-45e1-95e7-314d99ebfb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:10.958254 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516997cc-56d6-4e8f-bb54-240bc7c26fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:10.959227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516997cc-56d6-4e8f-bb54-240bc7c26fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:10.960111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516997cc-56d6-4e8f-bb54-240bc7c26fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:10.960450 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516997cc-56d6-4e8f-bb54-240bc7c26fd8 None admin] PATH_INFO: `/v3/users/99c293e528654716abf3be17db345acb` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:10.963820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6d8cdacd-3a2e-45e1-95e7-314d99ebfb14 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:10.965291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6d8cdacd-3a2e-45e1-95e7-314d99ebfb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:10.994750 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516997cc-56d6-4e8f-bb54-240bc7c26fd8 None admin] RBAC: Authorizing `identity:get_user(user_id=99c293e528654716abf3be17db345acb)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:10.997677 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516997cc-56d6-4e8f-bb54-240bc7c26fd8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:10.999834 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 58/116] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:10 2022] GET /identity/v3/users/99c293e528654716abf3be17db345acb => generated 289 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:11.001001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516997cc-56d6-4e8f-bb54-240bc7c26fd8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:11.080262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 59/117] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:11.084980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cdacd-3a2e-45e1-95e7-314d99ebfb14 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:11.161931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516997cc-56d6-4e8f-bb54-240bc7c26fd8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:11.226317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505a83f-fbb1-438d-a8dc-0e9a037135ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:11.227129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a83f-fbb1-438d-a8dc-0e9a037135ba None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:11.227442 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a83f-fbb1-438d-a8dc-0e9a037135ba None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:11.229137 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a83f-fbb1-438d-a8dc-0e9a037135ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:11.253980 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505a83f-fbb1-438d-a8dc-0e9a037135ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:11.264648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cdacd-3a2e-45e1-95e7-314d99ebfb14 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-1505a83f-fbb1-438d-a8dc-0e9a037135ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:11.274669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:11.276299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:11.277854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:11.277854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:11.277854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:11.277854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:11.277854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:11.277854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:11.277854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:11.277854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:11.277854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:11.277854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:11.277854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:11.277854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:11.281052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 59/118] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:53:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:11.281052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505a83f-fbb1-438d-a8dc-0e9a037135ba None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:11.315660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:11.344180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17724b4a-6fbd-4c35-b735-0a441a18f81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:11.345111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17724b4a-6fbd-4c35-b735-0a441a18f81e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:11.345403 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17724b4a-6fbd-4c35-b735-0a441a18f81e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:11.345689 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17724b4a-6fbd-4c35-b735-0a441a18f81e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:11.355829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:11.422739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505a83f-fbb1-438d-a8dc-0e9a037135ba None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:11.435155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17724b4a-6fbd-4c35-b735-0a441a18f81e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:11.436942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17724b4a-6fbd-4c35-b735-0a441a18f81e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:11.460638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed1612-d6d1-442d-945b-b0b8205cdfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:11.461912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed1612-d6d1-442d-945b-b0b8205cdfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:11.462213 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed1612-d6d1-442d-945b-b0b8205cdfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:11.462502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed1612-d6d1-442d-945b-b0b8205cdfa3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:11.474203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed1612-d6d1-442d-945b-b0b8205cdfa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-59ed1612-d6d1-442d-945b-b0b8205cdfa3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:11.482579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:11.484304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:11.485840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 60/119] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:11.487064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed1612-d6d1-442d-945b-b0b8205cdfa3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:11.547477 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 60/120] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:11.552983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b299db-ef8a-4156-bd89-0cf56198f55b None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:11.553913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b299db-ef8a-4156-bd89-0cf56198f55b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:11.554200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b299db-ef8a-4156-bd89-0cf56198f55b None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:11.555860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 61/121] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:11.556359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b299db-ef8a-4156-bd89-0cf56198f55b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:11.581569 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:11.603211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:11.608540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed1612-d6d1-442d-945b-b0b8205cdfa3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:11.648159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218aa322-efc4-4602-b117-587d147ba587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:11.648851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aa322-efc4-4602-b117-587d147ba587 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:11.649145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aa322-efc4-4602-b117-587d147ba587 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:11.649572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aa322-efc4-4602-b117-587d147ba587 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:11.650780 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218aa322-efc4-4602-b117-587d147ba587 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:11.650999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218aa322-efc4-4602-b117-587d147ba587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:11.652500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218aa322-efc4-4602-b117-587d147ba587 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:11.666883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 61/122] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:11.666883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218aa322-efc4-4602-b117-587d147ba587 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:11.676705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b299db-ef8a-4156-bd89-0cf56198f55b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:11.722185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90f73d-a3b5-4a12-8cc0-ef31359a6508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:11.723230 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90f73d-a3b5-4a12-8cc0-ef31359a6508 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:11.723628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90f73d-a3b5-4a12-8cc0-ef31359a6508 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:11.723921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90f73d-a3b5-4a12-8cc0-ef31359a6508 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:11.725573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90f73d-a3b5-4a12-8cc0-ef31359a6508 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:11.726696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90f73d-a3b5-4a12-8cc0-ef31359a6508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:11.729649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90f73d-a3b5-4a12-8cc0-ef31359a6508 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:11.744188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 62/123] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:11.745570 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf06ea-5c69-416a-bcb2-a1cb52b68407 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:11.745928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf06ea-5c69-416a-bcb2-a1cb52b68407 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:11.746321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf06ea-5c69-416a-bcb2-a1cb52b68407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:11.767608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-13cf06ea-5c69-416a-bcb2-a1cb52b68407 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:11.768923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-13cf06ea-5c69-416a-bcb2-a1cb52b68407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:11.805807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218aa322-efc4-4602-b117-587d147ba587 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:11.845159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcb2b5-3372-47eb-bfa9-363c95b06fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:11.846037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcb2b5-3372-47eb-bfa9-363c95b06fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:11.846339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcb2b5-3372-47eb-bfa9-363c95b06fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:11.846671 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcb2b5-3372-47eb-bfa9-363c95b06fe0 None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:11.847948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcb2b5-3372-47eb-bfa9-363c95b06fe0 None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:11.849007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 63/124] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:11.849698 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf06ea-5c69-416a-bcb2-a1cb52b68407 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:11.855215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcb2b5-3372-47eb-bfa9-363c95b06fe0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:11.857053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 62/125] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:11 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:11.859030 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcb2b5-3372-47eb-bfa9-363c95b06fe0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:11.867741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:11.911696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:11.991636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf06ea-5c69-416a-bcb2-a1cb52b68407 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:12.000009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcb2b5-3372-47eb-bfa9-363c95b06fe0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:12.035296 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db6f48-d013-4fbb-bc10-ed8de717ed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:12.036235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db6f48-d013-4fbb-bc10-ed8de717ed8f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:12.036513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db6f48-d013-4fbb-bc10-ed8de717ed8f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:12.036797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db6f48-d013-4fbb-bc10-ed8de717ed8f None admin] PATH_INFO: `/v3/users/5e6846265bb648f1b7034eadef52b2d8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:12.038530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db6f48-d013-4fbb-bc10-ed8de717ed8f None admin] RBAC: Authorizing `identity:get_user(user_id=5e6846265bb648f1b7034eadef52b2d8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:12.039341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e1003-d8d7-4c84-80f7-32ac11484737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:12.040045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db6f48-d013-4fbb-bc10-ed8de717ed8f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:12.040704 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e1003-d8d7-4c84-80f7-32ac11484737 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:12.041184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e1003-d8d7-4c84-80f7-32ac11484737 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:12.041658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e1003-d8d7-4c84-80f7-32ac11484737 None admin] PATH_INFO: `/v3/users/eae05ea06648449e9eda17d7c9a3f3fb` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:12.041757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 63/126] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:11 2022] GET /identity/v3/users/5e6846265bb648f1b7034eadef52b2d8 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:12.043171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e1003-d8d7-4c84-80f7-32ac11484737 None admin] RBAC: Authorizing `identity:get_user(user_id=eae05ea06648449e9eda17d7c9a3f3fb)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:12.047219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e1003-d8d7-4c84-80f7-32ac11484737 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:12.048820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db6f48-d013-4fbb-bc10-ed8de717ed8f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:12.049830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 64/127] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:11 2022] GET /identity/v3/users/eae05ea06648449e9eda17d7c9a3f3fb => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:12.050818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e1003-d8d7-4c84-80f7-32ac11484737 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:12.150918 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:12.158927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db6f48-d013-4fbb-bc10-ed8de717ed8f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:12.166081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:12.198640 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e1003-d8d7-4c84-80f7-32ac11484737 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:12.214934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7ec7f-c5d4-4501-a302-d2320b05026c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:12.216402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7ec7f-c5d4-4501-a302-d2320b05026c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:12.217009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7ec7f-c5d4-4501-a302-d2320b05026c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:12.217444 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7ec7f-c5d4-4501-a302-d2320b05026c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:12.218710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7ec7f-c5d4-4501-a302-d2320b05026c None admin] RBAC: Adding query filter params (role.id=b628b5b418f143f7a1d72f29749e04fd, scope.project.id=cc744332e3b04129956875355de3f205, user.id=99c293e528654716abf3be17db345acb) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:12.219563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7ec7f-c5d4-4501-a302-d2320b05026c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:12.221193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7ec7f-c5d4-4501-a302-d2320b05026c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:12.230311 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 64/128] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:12 2022] GET /identity/v3/role_assignments?user.id=99c293e528654716abf3be17db345acb&scope.project.id=cc744332e3b04129956875355de3f205&role.id=b628b5b418f143f7a1d72f29749e04fd => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:12.231636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7ec7f-c5d4-4501-a302-d2320b05026c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:12.238198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec09f47-6516-4c26-ae9f-f100765f1497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:12.239179 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec09f47-6516-4c26-ae9f-f100765f1497 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:12.239650 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec09f47-6516-4c26-ae9f-f100765f1497 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:12.240036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec09f47-6516-4c26-ae9f-f100765f1497 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:12.252916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec09f47-6516-4c26-ae9f-f100765f1497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-4ec09f47-6516-4c26-ae9f-f100765f1497 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:12.261353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:12.263003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:12.263993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:12.263993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:12.263993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:12.263993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:12.263993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:12.263993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:12.263993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:12.263993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:12.263993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:12.263993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:12.263993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:12.263993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:12.265516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 65/129] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:12.271645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec09f47-6516-4c26-ae9f-f100765f1497 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:12.390999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7ec7f-c5d4-4501-a302-d2320b05026c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:12.391855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec09f47-6516-4c26-ae9f-f100765f1497 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:12.429431 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f634998-72de-4f12-a9b7-bcd37bf10ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:12.430386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f634998-72de-4f12-a9b7-bcd37bf10ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:12.430915 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f634998-72de-4f12-a9b7-bcd37bf10ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:12.431185 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f634998-72de-4f12-a9b7-bcd37bf10ad9 None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:12.432870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f634998-72de-4f12-a9b7-bcd37bf10ad9 None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:12.434462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f634998-72de-4f12-a9b7-bcd37bf10ad9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:12.436358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 65/130] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:12 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:12.437354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f634998-72de-4f12-a9b7-bcd37bf10ad9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:12.440149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286e51a-f834-4db8-b1c5-de1f442011de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:12.441284 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286e51a-f834-4db8-b1c5-de1f442011de None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:12.441797 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286e51a-f834-4db8-b1c5-de1f442011de None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:12.442274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286e51a-f834-4db8-b1c5-de1f442011de None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:12.444365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286e51a-f834-4db8-b1c5-de1f442011de None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:12.447506 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286e51a-f834-4db8-b1c5-de1f442011de None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:12.449670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 66/131] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:12 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:12.450671 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286e51a-f834-4db8-b1c5-de1f442011de None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:12.468806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:12.479938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:12.581306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f634998-72de-4f12-a9b7-bcd37bf10ad9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:12.628326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4faad3-5458-4810-8f7f-a231f0319af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:12.629030 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4faad3-5458-4810-8f7f-a231f0319af7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:12.629320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4faad3-5458-4810-8f7f-a231f0319af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:12.629631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4faad3-5458-4810-8f7f-a231f0319af7 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:12.630408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4faad3-5458-4810-8f7f-a231f0319af7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:12.630723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4faad3-5458-4810-8f7f-a231f0319af7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:12.632221 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4faad3-5458-4810-8f7f-a231f0319af7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:12.643239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 66/132] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:12.644112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4faad3-5458-4810-8f7f-a231f0319af7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:12.680551 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286e51a-f834-4db8-b1c5-de1f442011de None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:12.717280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a9438-03d7-4ae9-a78c-81d1de2717d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:12.718244 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a9438-03d7-4ae9-a78c-81d1de2717d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:12.718534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a9438-03d7-4ae9-a78c-81d1de2717d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:12.718868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a9438-03d7-4ae9-a78c-81d1de2717d8 None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205/users/5e6846265bb648f1b7034eadef52b2d8/roles/ff3de46e4867421f990563b41a305d4d` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:12.721307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a9438-03d7-4ae9-a78c-81d1de2717d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc744332e3b04129956875355de3f205, user_id=5e6846265bb648f1b7034eadef52b2d8, role_id=ff3de46e4867421f990563b41a305d4d)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:12.723037 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a9438-03d7-4ae9-a78c-81d1de2717d8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:12.747697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 67/133] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:53:12 2022] PUT /identity/v3/projects/cc744332e3b04129956875355de3f205/users/5e6846265bb648f1b7034eadef52b2d8/roles/ff3de46e4867421f990563b41a305d4d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:12.748899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a9438-03d7-4ae9-a78c-81d1de2717d8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:12.791700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:12.808527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4faad3-5458-4810-8f7f-a231f0319af7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:12.876113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:12.900223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d3461-b281-4a52-a342-6876950e65c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:12.901168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d3461-b281-4a52-a342-6876950e65c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:12.901448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d3461-b281-4a52-a342-6876950e65c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:12.901737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d3461-b281-4a52-a342-6876950e65c6 None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205/users/eae05ea06648449e9eda17d7c9a3f3fb/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:12.903892 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d3461-b281-4a52-a342-6876950e65c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc744332e3b04129956875355de3f205, user_id=eae05ea06648449e9eda17d7c9a3f3fb, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:12.905409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d3461-b281-4a52-a342-6876950e65c6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:12.942647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a9438-03d7-4ae9-a78c-81d1de2717d8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:12.946056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 67/134] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:53:12 2022] PUT /identity/v3/projects/cc744332e3b04129956875355de3f205/users/eae05ea06648449e9eda17d7c9a3f3fb/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:12.992564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7594c2e-09e0-4d27-a5ce-d4099dffd16e None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:12.992877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7594c2e-09e0-4d27-a5ce-d4099dffd16e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:12.993155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7594c2e-09e0-4d27-a5ce-d4099dffd16e None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:12.994946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 68/135] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:12.999528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4514f-5343-4329-b88b-7a0369e8a788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:13.000502 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4514f-5343-4329-b88b-7a0369e8a788 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:13.000786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d88e2-9e88-44f3-8eca-886a500d437e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:13.000981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4514f-5343-4329-b88b-7a0369e8a788 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:13.001063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4514f-5343-4329-b88b-7a0369e8a788 None admin] PATH_INFO: `/v3/users/5e6846265bb648f1b7034eadef52b2d8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:13.001135 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d88e2-9e88-44f3-8eca-886a500d437e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:13.001367 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d88e2-9e88-44f3-8eca-886a500d437e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:13.003896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4514f-5343-4329-b88b-7a0369e8a788 None admin] RBAC: Authorizing `identity:get_user(user_id=5e6846265bb648f1b7034eadef52b2d8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:13.005527 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4514f-5343-4329-b88b-7a0369e8a788 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:13.007802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 68/136] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:12 2022] GET /identity/v3/users/5e6846265bb648f1b7034eadef52b2d8 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:13.013830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4514f-5343-4329-b88b-7a0369e8a788 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:13.036674 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e06d88e2-9e88-44f3-8eca-886a500d437e None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:13.038534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e06d88e2-9e88-44f3-8eca-886a500d437e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:13.177003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4514f-5343-4329-b88b-7a0369e8a788 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:13.196925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 69/137] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:13.232553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d88e2-9e88-44f3-8eca-886a500d437e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:13.235651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ca922-a1a9-4044-b9e5-05993fdc4448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:13.236948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ca922-a1a9-4044-b9e5-05993fdc4448 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:13.237306 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ca922-a1a9-4044-b9e5-05993fdc4448 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:13.237666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ca922-a1a9-4044-b9e5-05993fdc4448 None admin] PATH_INFO: `/v3/projects/d152cbf8b76149db89069b619ae0f5c1` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:13.240200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ca922-a1a9-4044-b9e5-05993fdc4448 None admin] RBAC: Authorizing `identity:get_project(project_id=d152cbf8b76149db89069b619ae0f5c1)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:13.242210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ca922-a1a9-4044-b9e5-05993fdc4448 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:13.244410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 69/138] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:13 2022] GET /identity/v3/projects/d152cbf8b76149db89069b619ae0f5c1 => generated 312 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:13.251636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ca922-a1a9-4044-b9e5-05993fdc4448 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:13.255993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:13.367856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ca922-a1a9-4044-b9e5-05993fdc4448 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:13.404203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:13.407948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4238fe-45d4-420f-916e-ec90ddcea7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:13.409125 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4238fe-45d4-420f-916e-ec90ddcea7fe None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:13.409634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4238fe-45d4-420f-916e-ec90ddcea7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:13.410154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4238fe-45d4-420f-916e-ec90ddcea7fe None admin] PATH_INFO: `/v3/projects/d152cbf8b76149db89069b619ae0f5c1/users/5e6846265bb648f1b7034eadef52b2d8/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:13.413917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4238fe-45d4-420f-916e-ec90ddcea7fe None admin] RBAC: Authorizing `identity:create_grant(project_id=d152cbf8b76149db89069b619ae0f5c1, user_id=5e6846265bb648f1b7034eadef52b2d8, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:13.415761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4238fe-45d4-420f-916e-ec90ddcea7fe None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:13.431003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 70/139] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:53:13 2022] PUT /identity/v3/projects/d152cbf8b76149db89069b619ae0f5c1/users/5e6846265bb648f1b7034eadef52b2d8/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:13.438914 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d88e2-9e88-44f3-8eca-886a500d437e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:13.479997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1a475-f66c-446e-8455-530bd28248d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:13.480879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1a475-f66c-446e-8455-530bd28248d1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:13.481176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1a475-f66c-446e-8455-530bd28248d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:13.481450 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1a475-f66c-446e-8455-530bd28248d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:13.493927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1a475-f66c-446e-8455-530bd28248d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-b3c1a475-f66c-446e-8455-530bd28248d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:13.502012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:13.502925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:13.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 70/140] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:13.509796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4238fe-45d4-420f-916e-ec90ddcea7fe None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:13.607682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:13.655973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4238fe-45d4-420f-916e-ec90ddcea7fe None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:13.692774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92b4c3-6a87-4989-8b6a-55c7a94b558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:13.693394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92b4c3-6a87-4989-8b6a-55c7a94b558b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:13.694266 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92b4c3-6a87-4989-8b6a-55c7a94b558b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:13.694563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92b4c3-6a87-4989-8b6a-55c7a94b558b None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:13.695335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92b4c3-6a87-4989-8b6a-55c7a94b558b None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:13.695675 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92b4c3-6a87-4989-8b6a-55c7a94b558b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:13.697100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92b4c3-6a87-4989-8b6a-55c7a94b558b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:13.706279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 71/141] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:13.711997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1a475-f66c-446e-8455-530bd28248d1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:13.860551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1a475-f66c-446e-8455-530bd28248d1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:13.905552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37ceb8-12de-4307-9900-15983ffa3f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:13.906488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37ceb8-12de-4307-9900-15983ffa3f18 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:13.906821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37ceb8-12de-4307-9900-15983ffa3f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:13.907102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37ceb8-12de-4307-9900-15983ffa3f18 None admin] PATH_INFO: `/v3/users/5e6846265bb648f1b7034eadef52b2d8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:13.908345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37ceb8-12de-4307-9900-15983ffa3f18 None admin] RBAC: Authorizing `identity:get_user(user_id=5e6846265bb648f1b7034eadef52b2d8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:13.909897 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37ceb8-12de-4307-9900-15983ffa3f18 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:13.911858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 71/142] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:13 2022] GET /identity/v3/users/5e6846265bb648f1b7034eadef52b2d8 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:13.917316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92b4c3-6a87-4989-8b6a-55c7a94b558b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:13.946554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a429c-ae0d-4609-9483-edef394fa33d None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:13.947044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a429c-ae0d-4609-9483-edef394fa33d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:13.947044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a429c-ae0d-4609-9483-edef394fa33d None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:13.948524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 72/143] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:53:13.978116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee24513-90d5-489a-bf3e-da12f2768e11 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:13.978411 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee24513-90d5-489a-bf3e-da12f2768e11 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:13.978676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee24513-90d5-489a-bf3e-da12f2768e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:14.016565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7ee24513-90d5-489a-bf3e-da12f2768e11 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:14.017831 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7ee24513-90d5-489a-bf3e-da12f2768e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:14.020053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:14.039672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92b4c3-6a87-4989-8b6a-55c7a94b558b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:14.082947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:14.088174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ccf08-6a19-4928-9233-008a107f5f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:14.089276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ccf08-6a19-4928-9233-008a107f5f90 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:14.089772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ccf08-6a19-4928-9233-008a107f5f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:14.090277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ccf08-6a19-4928-9233-008a107f5f90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:14.154026 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 73/144] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:14.213731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ccf08-6a19-4928-9233-008a107f5f90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:14.219743 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ccf08-6a19-4928-9233-008a107f5f90 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:14.268786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee24513-90d5-489a-bf3e-da12f2768e11 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:14.420800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee24513-90d5-489a-bf3e-da12f2768e11 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:14.456018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 72/145] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:14.465100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ccf08-6a19-4928-9233-008a107f5f90 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:14.506464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:14.507485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fb770-068f-4e09-abdf-f4c7833c4207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:14.508561 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fb770-068f-4e09-abdf-f4c7833c4207 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:14.509101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fb770-068f-4e09-abdf-f4c7833c4207 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:14.509576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fb770-068f-4e09-abdf-f4c7833c4207 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:14.518327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fb770-068f-4e09-abdf-f4c7833c4207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:14.520161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fb770-068f-4e09-abdf-f4c7833c4207 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:14.522757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 74/146] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:14.528500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fb770-068f-4e09-abdf-f4c7833c4207 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:14.549644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:14.641769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ccf08-6a19-4928-9233-008a107f5f90 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:14.684455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fb770-068f-4e09-abdf-f4c7833c4207 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:14.744297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c496ed4-871a-4841-bffc-f6a9072b2688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:14.745484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c496ed4-871a-4841-bffc-f6a9072b2688 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:14.745784 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c496ed4-871a-4841-bffc-f6a9072b2688 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:14.746146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c496ed4-871a-4841-bffc-f6a9072b2688 None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:14.747603 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c496ed4-871a-4841-bffc-f6a9072b2688 None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:14.749075 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c496ed4-871a-4841-bffc-f6a9072b2688 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:14.751201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 73/147] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:14 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:14.752085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d178dd-7f80-41be-8b84-006811bca57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:14.752772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d178dd-7f80-41be-8b84-006811bca57f None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:14.753056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d178dd-7f80-41be-8b84-006811bca57f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:14.753335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d178dd-7f80-41be-8b84-006811bca57f None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:14.754069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d178dd-7f80-41be-8b84-006811bca57f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:14.755614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d178dd-7f80-41be-8b84-006811bca57f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:14.759590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c496ed4-871a-4841-bffc-f6a9072b2688 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:14.784498 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 75/148] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:53:14 2022] POST /identity/v3/users => generated 283 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:53:14.926566 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:14.936210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c496ed4-871a-4841-bffc-f6a9072b2688 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:14.981683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e7c38-6c41-451f-a659-412f993ab35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:14.982366 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e7c38-6c41-451f-a659-412f993ab35d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:14.982656 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e7c38-6c41-451f-a659-412f993ab35d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:14.982938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e7c38-6c41-451f-a659-412f993ab35d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:14.983827 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e7c38-6c41-451f-a659-412f993ab35d None admin] RBAC: Adding query filter params (role.id=9a562f9a0cad4304852f3183eef398d2, scope.project.id=cc744332e3b04129956875355de3f205, user.id=5e6846265bb648f1b7034eadef52b2d8) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:14.984147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e7c38-6c41-451f-a659-412f993ab35d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:14.985627 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e7c38-6c41-451f-a659-412f993ab35d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:14.992693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 74/149] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:14 2022] GET /identity/v3/role_assignments?user.id=5e6846265bb648f1b7034eadef52b2d8&scope.project.id=cc744332e3b04129956875355de3f205&role.id=9a562f9a0cad4304852f3183eef398d2 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:15.451938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688426be-e5da-4e50-b53f-a1d0b995b662 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:15.452419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688426be-e5da-4e50-b53f-a1d0b995b662 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:15.452419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688426be-e5da-4e50-b53f-a1d0b995b662 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:15.454014 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 76/150] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:15.463585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640ecb4-a67f-400c-af14-bbf3e9147ca1 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:15.464086 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640ecb4-a67f-400c-af14-bbf3e9147ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:15.464161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640ecb4-a67f-400c-af14-bbf3e9147ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:15.484018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2640ecb4-a67f-400c-af14-bbf3e9147ca1 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:15.485221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2640ecb4-a67f-400c-af14-bbf3e9147ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:15.569094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 75/151] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:15.591843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688426be-e5da-4e50-b53f-a1d0b995b662 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:15.661729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:15.706173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688426be-e5da-4e50-b53f-a1d0b995b662 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:15.742315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52584d9f-903b-41f0-8c1a-bf12e595ad6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:15.743130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52584d9f-903b-41f0-8c1a-bf12e595ad6d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:15.743442 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52584d9f-903b-41f0-8c1a-bf12e595ad6d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:15.743720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52584d9f-903b-41f0-8c1a-bf12e595ad6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:15.755244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52584d9f-903b-41f0-8c1a-bf12e595ad6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-52584d9f-903b-41f0-8c1a-bf12e595ad6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:15.763318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:15.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:15.765047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 77/152] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:15.770596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640ecb4-a67f-400c-af14-bbf3e9147ca1 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:15.875808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:15.885680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c25dd-7a0a-4652-8dd5-a980a5aea662 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:15.885971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c25dd-7a0a-4652-8dd5-a980a5aea662 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:15.886247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c25dd-7a0a-4652-8dd5-a980a5aea662 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:15.887880 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 78/153] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:15.893565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a611d-4f4f-477d-927e-98688b7ae0f6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:15.893879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a611d-4f4f-477d-927e-98688b7ae0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:15.894126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a611d-4f4f-477d-927e-98688b7ae0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:15.894673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640ecb4-a67f-400c-af14-bbf3e9147ca1 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:15.922005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-601a611d-4f4f-477d-927e-98688b7ae0f6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:15.923616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-601a611d-4f4f-477d-927e-98688b7ae0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:15.976279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c08e0-ff4a-4de1-a5db-b70262b1461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:15.976899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c08e0-ff4a-4de1-a5db-b70262b1461a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:15.977187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c08e0-ff4a-4de1-a5db-b70262b1461a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:15.977465 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c08e0-ff4a-4de1-a5db-b70262b1461a None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:15.978240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c08e0-ff4a-4de1-a5db-b70262b1461a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:15.978551 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c08e0-ff4a-4de1-a5db-b70262b1461a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:15.979986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c08e0-ff4a-4de1-a5db-b70262b1461a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:15.995186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 76/154] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:15.996235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 79/155] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:16.008890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c08e0-ff4a-4de1-a5db-b70262b1461a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:16.024388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a611d-4f4f-477d-927e-98688b7ae0f6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:16.078378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:16.129650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a611d-4f4f-477d-927e-98688b7ae0f6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:16.172832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfada1-e24e-489e-9bc3-fd2797f1e221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:16.173669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfada1-e24e-489e-9bc3-fd2797f1e221 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:16.173953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfada1-e24e-489e-9bc3-fd2797f1e221 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:16.174232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfada1-e24e-489e-9bc3-fd2797f1e221 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:16.188653 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfada1-e24e-489e-9bc3-fd2797f1e221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-e3cfada1-e24e-489e-9bc3-fd2797f1e221 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:16.199502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:16.200487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:16.201307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 80/156] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:53:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:16.204811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6450-f8f3-4d42-9012-5e23e4d7670a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:16.205096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6450-f8f3-4d42-9012-5e23e4d7670a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:16.205514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6450-f8f3-4d42-9012-5e23e4d7670a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:16.211857 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 81/157] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:16.211857 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b6450-f8f3-4d42-9012-5e23e4d7670a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:16.218588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c08e0-ff4a-4de1-a5db-b70262b1461a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:16.280418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd3486-9eea-46f8-a116-c5584f3c0a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:16.281768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd3486-9eea-46f8-a116-c5584f3c0a1f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:16.282661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd3486-9eea-46f8-a116-c5584f3c0a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:16.283722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd3486-9eea-46f8-a116-c5584f3c0a1f None admin] PATH_INFO: `/v3/users/99c293e528654716abf3be17db345acb` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:16.285426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd3486-9eea-46f8-a116-c5584f3c0a1f None admin] RBAC: Authorizing `identity:get_user(user_id=99c293e528654716abf3be17db345acb)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:16.289661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd3486-9eea-46f8-a116-c5584f3c0a1f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:16.297429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 77/158] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:16 2022] GET /identity/v3/users/99c293e528654716abf3be17db345acb => generated 289 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:16.299186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39a0b8-7587-4997-8016-ac21a54e2ce0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:16.300043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39a0b8-7587-4997-8016-ac21a54e2ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:16.300536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39a0b8-7587-4997-8016-ac21a54e2ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:16.320362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0e39a0b8-7587-4997-8016-ac21a54e2ce0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:16.321928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0e39a0b8-7587-4997-8016-ac21a54e2ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:16.347755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:16.369209 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b6450-f8f3-4d42-9012-5e23e4d7670a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:16.392839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:16.399416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 78/159] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:16.400922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39a0b8-7587-4997-8016-ac21a54e2ce0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:16.453379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618de0cd-1954-4816-8a67-ce3e40c050ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:16.453379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618de0cd-1954-4816-8a67-ce3e40c050ad None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:16.453379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618de0cd-1954-4816-8a67-ce3e40c050ad None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:16.453379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618de0cd-1954-4816-8a67-ce3e40c050ad None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:16.454098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618de0cd-1954-4816-8a67-ce3e40c050ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:16.454500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618de0cd-1954-4816-8a67-ce3e40c050ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:16.456046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618de0cd-1954-4816-8a67-ce3e40c050ad None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:16.474692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 82/160] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:16.475723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618de0cd-1954-4816-8a67-ce3e40c050ad None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:16.535222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39a0b8-7587-4997-8016-ac21a54e2ce0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:16.572997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4108ec9-b817-49e0-ae85-acc3b8646fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:16.574333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4108ec9-b817-49e0-ae85-acc3b8646fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:16.574919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4108ec9-b817-49e0-ae85-acc3b8646fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:16.575428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4108ec9-b817-49e0-ae85-acc3b8646fd4 None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:16.576865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4108ec9-b817-49e0-ae85-acc3b8646fd4 None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:16.578519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4108ec9-b817-49e0-ae85-acc3b8646fd4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:16.580606 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 79/161] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:16 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:16.581576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4108ec9-b817-49e0-ae85-acc3b8646fd4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:16.603110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:16.695864 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4108ec9-b817-49e0-ae85-acc3b8646fd4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:16.732944 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618de0cd-1954-4816-8a67-ce3e40c050ad None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:16.755098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb41d4-c20c-466c-ab24-6e9f392f422c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:16.758876 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb41d4-c20c-466c-ab24-6e9f392f422c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:16.760853 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb41d4-c20c-466c-ab24-6e9f392f422c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:16.762019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb41d4-c20c-466c-ab24-6e9f392f422c None admin] PATH_INFO: `/v3/users/eae05ea06648449e9eda17d7c9a3f3fb` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:16.765538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb41d4-c20c-466c-ab24-6e9f392f422c None admin] RBAC: Authorizing `identity:get_user(user_id=eae05ea06648449e9eda17d7c9a3f3fb)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:16.768693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb41d4-c20c-466c-ab24-6e9f392f422c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:16.771766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 80/162] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:16 2022] GET /identity/v3/users/eae05ea06648449e9eda17d7c9a3f3fb => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:16.775369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf26b5-d716-4591-ad6b-dd9c7a2aada8 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:16.779935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf26b5-d716-4591-ad6b-dd9c7a2aada8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:16.780520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf26b5-d716-4591-ad6b-dd9c7a2aada8 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:16.782353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 81/163] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:16.784800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf26b5-d716-4591-ad6b-dd9c7a2aada8 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:16.811840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75995d53-082e-499b-8007-63dfe328d37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:16.813017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75995d53-082e-499b-8007-63dfe328d37c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:16.814580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75995d53-082e-499b-8007-63dfe328d37c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:16.815067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75995d53-082e-499b-8007-63dfe328d37c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:16.837940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75995d53-082e-499b-8007-63dfe328d37c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-75995d53-082e-499b-8007-63dfe328d37c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:16.846415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:16.847295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 83/164] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:16.848757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75995d53-082e-499b-8007-63dfe328d37c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:16.891983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:16.969564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:16.991523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf26b5-d716-4591-ad6b-dd9c7a2aada8 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:17.017518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75995d53-082e-499b-8007-63dfe328d37c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:17.037195 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065838ab-ce8e-44c9-878f-6b6498a862c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:17.038647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065838ab-ce8e-44c9-878f-6b6498a862c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:17.038930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065838ab-ce8e-44c9-878f-6b6498a862c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:17.039213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065838ab-ce8e-44c9-878f-6b6498a862c1 None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205/users/99c293e528654716abf3be17db345acb/roles/b628b5b418f143f7a1d72f29749e04fd` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:17.044028 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065838ab-ce8e-44c9-878f-6b6498a862c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc744332e3b04129956875355de3f205, user_id=99c293e528654716abf3be17db345acb, role_id=b628b5b418f143f7a1d72f29749e04fd)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:17.048002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065838ab-ce8e-44c9-878f-6b6498a862c1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:17.075447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 82/165] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:53:16 2022] PUT /identity/v3/projects/cc744332e3b04129956875355de3f205/users/99c293e528654716abf3be17db345acb/roles/b628b5b418f143f7a1d72f29749e04fd => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:17.076814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9bbf2-6ff0-480d-a066-369284f456f5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:17.077079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9bbf2-6ff0-480d-a066-369284f456f5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:17.077365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9bbf2-6ff0-480d-a066-369284f456f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:17.095145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437507e-8fda-424a-b14b-3205ec40b909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:17.095895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437507e-8fda-424a-b14b-3205ec40b909 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:17.096222 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437507e-8fda-424a-b14b-3205ec40b909 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:17.096574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437507e-8fda-424a-b14b-3205ec40b909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:17.112667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-09d9bbf2-6ff0-480d-a066-369284f456f5 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:17.114293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-09d9bbf2-6ff0-480d-a066-369284f456f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:17.227198 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437507e-8fda-424a-b14b-3205ec40b909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:17.230418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437507e-8fda-424a-b14b-3205ec40b909 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:17.264985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 83/166] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:17.264985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9bbf2-6ff0-480d-a066-369284f456f5 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:17.317498 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:17.410896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 84/167] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:17.411313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:17.411607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437507e-8fda-424a-b14b-3205ec40b909 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:17.438209 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9bbf2-6ff0-480d-a066-369284f456f5 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:17.480322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b584b02-d044-4de6-8030-37a480da380d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:17.481290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b584b02-d044-4de6-8030-37a480da380d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:17.481583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b584b02-d044-4de6-8030-37a480da380d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:17.481894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b584b02-d044-4de6-8030-37a480da380d None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:17.482871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b584b02-d044-4de6-8030-37a480da380d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:17.483186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b584b02-d044-4de6-8030-37a480da380d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:17.485099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b584b02-d044-4de6-8030-37a480da380d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:17.501158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 84/168] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:17.502232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b584b02-d044-4de6-8030-37a480da380d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:17.589524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437507e-8fda-424a-b14b-3205ec40b909 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:17.627575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b584b02-d044-4de6-8030-37a480da380d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:17.670246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f0f2e-c132-42f6-8f58-257786b7321a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:17.671049 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f0f2e-c132-42f6-8f58-257786b7321a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:17.671336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f0f2e-c132-42f6-8f58-257786b7321a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:17.671650 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f0f2e-c132-42f6-8f58-257786b7321a None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:17.672425 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd944e3f-8f40-4f29-beaf-bf38290f98fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:17.672975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f0f2e-c132-42f6-8f58-257786b7321a None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:17.673777 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd944e3f-8f40-4f29-beaf-bf38290f98fb None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:17.674262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd944e3f-8f40-4f29-beaf-bf38290f98fb None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:17.674433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f0f2e-c132-42f6-8f58-257786b7321a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:17.674894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd944e3f-8f40-4f29-beaf-bf38290f98fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:17.676462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 85/169] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:17 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:17.677040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f0f2e-c132-42f6-8f58-257786b7321a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:17.679005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:17.683407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:17.702673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd944e3f-8f40-4f29-beaf-bf38290f98fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-bd944e3f-8f40-4f29-beaf-bf38290f98fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:17.719649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:17.720608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:17.721369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:17.721369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:17.721369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:17.721369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:17.721369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:17.721369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:17.721369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:17.721369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:17.721369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:17.721369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:17.721369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:17.721369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:17.724259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 85/170] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:17.725872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd944e3f-8f40-4f29-beaf-bf38290f98fb None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:17.803595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f0f2e-c132-42f6-8f58-257786b7321a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:17.846953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df264cda-d640-4bb8-a14f-095a4e88bf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:17.847924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df264cda-d640-4bb8-a14f-095a4e88bf73 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:17.848267 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df264cda-d640-4bb8-a14f-095a4e88bf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:17.848556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df264cda-d640-4bb8-a14f-095a4e88bf73 None admin] PATH_INFO: `/v3/users/5e6846265bb648f1b7034eadef52b2d8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:17.850702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df264cda-d640-4bb8-a14f-095a4e88bf73 None admin] RBAC: Authorizing `identity:get_user(user_id=5e6846265bb648f1b7034eadef52b2d8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:17.852504 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df264cda-d640-4bb8-a14f-095a4e88bf73 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:17.854406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 86/171] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:17 2022] GET /identity/v3/users/5e6846265bb648f1b7034eadef52b2d8 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:17.856410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df264cda-d640-4bb8-a14f-095a4e88bf73 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:17.929094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd944e3f-8f40-4f29-beaf-bf38290f98fb None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:17.944735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:17.981430 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df264cda-d640-4bb8-a14f-095a4e88bf73 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:18.008709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bfe67-5dd4-43c7-b4f6-464d296ed6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:18.009386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bfe67-5dd4-43c7-b4f6-464d296ed6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:18.009695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bfe67-5dd4-43c7-b4f6-464d296ed6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:18.009976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bfe67-5dd4-43c7-b4f6-464d296ed6c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:18.010822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bfe67-5dd4-43c7-b4f6-464d296ed6c0 None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=cc744332e3b04129956875355de3f205, user.id=eae05ea06648449e9eda17d7c9a3f3fb) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:18.011133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bfe67-5dd4-43c7-b4f6-464d296ed6c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:18.012639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bfe67-5dd4-43c7-b4f6-464d296ed6c0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:18.018641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d9f4f-af29-48db-a236-f23a3fec197c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:18.019333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d9f4f-af29-48db-a236-f23a3fec197c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:18.019657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d9f4f-af29-48db-a236-f23a3fec197c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:18.019948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d9f4f-af29-48db-a236-f23a3fec197c None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:18.020720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d9f4f-af29-48db-a236-f23a3fec197c None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:18.021040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d9f4f-af29-48db-a236-f23a3fec197c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:18.022507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d9f4f-af29-48db-a236-f23a3fec197c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:18.032286 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 87/172] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:18.033190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d9f4f-af29-48db-a236-f23a3fec197c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:18.035521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 86/173] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:17 2022] GET /identity/v3/role_assignments?user.id=eae05ea06648449e9eda17d7c9a3f3fb&scope.project.id=cc744332e3b04129956875355de3f205&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:18.045021 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bfe67-5dd4-43c7-b4f6-464d296ed6c0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:18.156059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d9f4f-af29-48db-a236-f23a3fec197c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:18.215231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fb25f-2386-4feb-93f2-78ca4f08780a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:18.216201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fb25f-2386-4feb-93f2-78ca4f08780a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:18.216495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fb25f-2386-4feb-93f2-78ca4f08780a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:18.216780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fb25f-2386-4feb-93f2-78ca4f08780a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:18.221276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:18.289761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:18.376533 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667fb25f-2386-4feb-93f2-78ca4f08780a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:18.378073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667fb25f-2386-4feb-93f2-78ca4f08780a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:18.441654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bfe67-5dd4-43c7-b4f6-464d296ed6c0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:18.544868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44330f1d-da9a-48d0-9703-e9b3c735e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:18.545703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44330f1d-da9a-48d0-9703-e9b3c735e228 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:18.545986 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44330f1d-da9a-48d0-9703-e9b3c735e228 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:18.546263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44330f1d-da9a-48d0-9703-e9b3c735e228 None admin] PATH_INFO: `/v3/users/5e6846265bb648f1b7034eadef52b2d8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:18.547524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44330f1d-da9a-48d0-9703-e9b3c735e228 None admin] RBAC: Authorizing `identity:get_user(user_id=5e6846265bb648f1b7034eadef52b2d8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:18.549006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44330f1d-da9a-48d0-9703-e9b3c735e228 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:18.550983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 87/174] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:18 2022] GET /identity/v3/users/5e6846265bb648f1b7034eadef52b2d8 => generated 275 bytes in 507 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:18.552657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31731acc-4787-47b6-a401-60a66f6b87bb None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:18.552657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31731acc-4787-47b6-a401-60a66f6b87bb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:18.552984 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31731acc-4787-47b6-a401-60a66f6b87bb None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:18.554184 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 88/175] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:18.563227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89071395-1b2e-49d0-a949-4a3336aa1fed None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:18.563596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89071395-1b2e-49d0-a949-4a3336aa1fed None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:18.563829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89071395-1b2e-49d0-a949-4a3336aa1fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:18.593250 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-89071395-1b2e-49d0-a949-4a3336aa1fed None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:18.594461 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-89071395-1b2e-49d0-a949-4a3336aa1fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:18.666661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 88/176] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:18.667655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fb25f-2386-4feb-93f2-78ca4f08780a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:18.736343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 89/177] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:18.737107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89071395-1b2e-49d0-a949-4a3336aa1fed None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:18.741415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:18.798321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:18.847895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89071395-1b2e-49d0-a949-4a3336aa1fed None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:18.887117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb797a-f9b8-40ba-ba27-162b6ec36ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:18.888912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb797a-f9b8-40ba-ba27-162b6ec36ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:18.889345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb797a-f9b8-40ba-ba27-162b6ec36ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:18.889761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb797a-f9b8-40ba-ba27-162b6ec36ce5 None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:18.891447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb797a-f9b8-40ba-ba27-162b6ec36ce5 None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:18.893329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb797a-f9b8-40ba-ba27-162b6ec36ce5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:18.895684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 90/178] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:18 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:18.896798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb797a-f9b8-40ba-ba27-162b6ec36ce5 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:18.909328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fb25f-2386-4feb-93f2-78ca4f08780a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:19.015174 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb797a-f9b8-40ba-ba27-162b6ec36ce5 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:19.036365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d8184-f2c2-4c63-ae6d-810299a75b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:19.037050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d8184-f2c2-4c63-ae6d-810299a75b75 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:19.037381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d8184-f2c2-4c63-ae6d-810299a75b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:19.037669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d8184-f2c2-4c63-ae6d-810299a75b75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:19.043812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:19.056856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841b882-2d2d-4967-a7f3-9520db11cf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:19.058782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841b882-2d2d-4967-a7f3-9520db11cf06 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:19.059273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841b882-2d2d-4967-a7f3-9520db11cf06 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:19.059719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841b882-2d2d-4967-a7f3-9520db11cf06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:19.075097 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841b882-2d2d-4967-a7f3-9520db11cf06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-6841b882-2d2d-4967-a7f3-9520db11cf06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:19.085967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:19.087001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:19.087959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:19.087959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:19.087959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:19.087959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:19.087959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:19.087959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:19.087959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:19.087959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:19.087959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:19.087959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:19.087959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:19.087959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:19.090503 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 91/179] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:53:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:19.091486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841b882-2d2d-4967-a7f3-9520db11cf06 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:19.120920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d8184-f2c2-4c63-ae6d-810299a75b75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:19.122505 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d8184-f2c2-4c63-ae6d-810299a75b75 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:19.157493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:19.219125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841b882-2d2d-4967-a7f3-9520db11cf06 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:19.253092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 89/180] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:19.255155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5048ef-979f-48fc-9e1a-6bb182fcf0fe None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:19.255755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5048ef-979f-48fc-9e1a-6bb182fcf0fe None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:19.256246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5048ef-979f-48fc-9e1a-6bb182fcf0fe None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:19.257926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c918f5-47cd-4a2d-bd9f-4f84ee8ee34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:19.258313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 90/181] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:19.259180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5048ef-979f-48fc-9e1a-6bb182fcf0fe None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:19.263754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c918f5-47cd-4a2d-bd9f-4f84ee8ee34f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:19.263894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c918f5-47cd-4a2d-bd9f-4f84ee8ee34f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:19.264215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c918f5-47cd-4a2d-bd9f-4f84ee8ee34f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:19.265188 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c918f5-47cd-4a2d-bd9f-4f84ee8ee34f None admin] RBAC: Adding query filter params (role.id=ff3de46e4867421f990563b41a305d4d, scope.project.id=cc744332e3b04129956875355de3f205, user.id=5e6846265bb648f1b7034eadef52b2d8) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:19.265582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c918f5-47cd-4a2d-bd9f-4f84ee8ee34f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:19.267153 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c918f5-47cd-4a2d-bd9f-4f84ee8ee34f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:19.278736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 92/182] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:19 2022] GET /identity/v3/role_assignments?user.id=5e6846265bb648f1b7034eadef52b2d8&scope.project.id=cc744332e3b04129956875355de3f205&role.id=ff3de46e4867421f990563b41a305d4d => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:19.279748 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c918f5-47cd-4a2d-bd9f-4f84ee8ee34f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:19.365186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:19.413799 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5048ef-979f-48fc-9e1a-6bb182fcf0fe None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:19.435904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c918f5-47cd-4a2d-bd9f-4f84ee8ee34f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:19.456648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ee94b-5928-4710-9b4f-1560facac547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:19.457332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ee94b-5928-4710-9b4f-1560facac547 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:19.457612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ee94b-5928-4710-9b4f-1560facac547 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:19.457886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ee94b-5928-4710-9b4f-1560facac547 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:19.459431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ee94b-5928-4710-9b4f-1560facac547 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:19.459431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ee94b-5928-4710-9b4f-1560facac547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:19.460935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ee94b-5928-4710-9b4f-1560facac547 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:19.462934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:19.474255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 91/183] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:19.478101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9d6da-e5f6-4a51-84dd-9df18598c86e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:19.478101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9d6da-e5f6-4a51-84dd-9df18598c86e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:19.478101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9d6da-e5f6-4a51-84dd-9df18598c86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:19.488439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a369ea-a4a1-4538-ab82-2c2aa503b83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:19.489610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a369ea-a4a1-4538-ab82-2c2aa503b83f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:19.490089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a369ea-a4a1-4538-ab82-2c2aa503b83f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:19.490436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a369ea-a4a1-4538-ab82-2c2aa503b83f None admin] PATH_INFO: `/v3/projects/d152cbf8b76149db89069b619ae0f5c1` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:19.492310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a369ea-a4a1-4538-ab82-2c2aa503b83f None admin] RBAC: Authorizing `identity:get_project(project_id=d152cbf8b76149db89069b619ae0f5c1)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:19.494406 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a369ea-a4a1-4538-ab82-2c2aa503b83f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:19.496527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 93/184] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:19 2022] GET /identity/v3/projects/d152cbf8b76149db89069b619ae0f5c1 => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:19.497776 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d15a55-a62d-42a0-8cce-79ff3c9c95df None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:19.498118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d15a55-a62d-42a0-8cce-79ff3c9c95df None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:19.498580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d15a55-a62d-42a0-8cce-79ff3c9c95df None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:19.499359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c7d9d6da-e5f6-4a51-84dd-9df18598c86e None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:19.500044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 94/185] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:19.500587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d15a55-a62d-42a0-8cce-79ff3c9c95df None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:19.500705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c7d9d6da-e5f6-4a51-84dd-9df18598c86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:19.589608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 92/186] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:19.590692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cb525-4734-4692-97ce-e65e05ff6744 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:19.590999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cb525-4734-4692-97ce-e65e05ff6744 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:19.591248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cb525-4734-4692-97ce-e65e05ff6744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:19.608573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-492cb525-4734-4692-97ce-e65e05ff6744 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:19.609255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-492cb525-4734-4692-97ce-e65e05ff6744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:19.648422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d15a55-a62d-42a0-8cce-79ff3c9c95df None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:19.666865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:19.688482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed69f66-55da-4830-a963-7a042ee7f845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:19.689378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed69f66-55da-4830-a963-7a042ee7f845 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:19.689663 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed69f66-55da-4830-a963-7a042ee7f845 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:19.689946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed69f66-55da-4830-a963-7a042ee7f845 None admin] PATH_INFO: `/v3/users/2ed9c364f8104907b156966f55ae4520` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:19.690785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 93/187] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:19.699418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cb525-4734-4692-97ce-e65e05ff6744 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:19.704447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed69f66-55da-4830-a963-7a042ee7f845 None admin] RBAC: Authorizing `identity:get_user(user_id=2ed9c364f8104907b156966f55ae4520)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:19.705942 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed69f66-55da-4830-a963-7a042ee7f845 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:19.708006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 95/188] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:19 2022] GET /identity/v3/users/2ed9c364f8104907b156966f55ae4520 => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:19.708798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed69f66-55da-4830-a963-7a042ee7f845 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:19.765989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:19.812999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cb525-4734-4692-97ce-e65e05ff6744 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:19.825866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed69f66-55da-4830-a963-7a042ee7f845 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:19.852594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd911d-fa23-442d-b4c2-488f100aaf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:19.853352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd911d-fa23-442d-b4c2-488f100aaf34 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:19.853637 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd911d-fa23-442d-b4c2-488f100aaf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:19.853912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd911d-fa23-442d-b4c2-488f100aaf34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:19.854744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd911d-fa23-442d-b4c2-488f100aaf34 None admin] RBAC: Adding query filter params (role.id=9a562f9a0cad4304852f3183eef398d2, scope.project.id=d152cbf8b76149db89069b619ae0f5c1, user.id=5e6846265bb648f1b7034eadef52b2d8) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:19.855062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd911d-fa23-442d-b4c2-488f100aaf34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:19.856582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd911d-fa23-442d-b4c2-488f100aaf34 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:19.862751 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 94/189] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:19 2022] GET /identity/v3/role_assignments?user.id=5e6846265bb648f1b7034eadef52b2d8&scope.project.id=d152cbf8b76149db89069b619ae0f5c1&role.id=9a562f9a0cad4304852f3183eef398d2 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:19.863737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037a090-4516-401e-8687-884cbe2338d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:19.864764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037a090-4516-401e-8687-884cbe2338d5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:19.864886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd911d-fa23-442d-b4c2-488f100aaf34 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:19.865340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037a090-4516-401e-8687-884cbe2338d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:19.865889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037a090-4516-401e-8687-884cbe2338d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:19.878728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037a090-4516-401e-8687-884cbe2338d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-2037a090-4516-401e-8687-884cbe2338d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:19.887588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:19.888533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:19.889346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:19.889346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:19.889346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:19.889346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:19.889346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:19.889346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:19.889346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:19.889346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:19.889346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:19.889346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:19.889346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:19.889346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:19.897755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 96/190] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:19.899111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037a090-4516-401e-8687-884cbe2338d5 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:19.939889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:20.031211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd911d-fa23-442d-b4c2-488f100aaf34 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:20.050164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037a090-4516-401e-8687-884cbe2338d5 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:20.056390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:20.075508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdca942-e4ea-46bf-81cd-85e67ea6173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:20.076530 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdca942-e4ea-46bf-81cd-85e67ea6173a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:20.076819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdca942-e4ea-46bf-81cd-85e67ea6173a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:20.077404 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdca942-e4ea-46bf-81cd-85e67ea6173a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:20.093998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9e6d6-1e0c-4653-823c-ecaf0533c02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:20.095127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9e6d6-1e0c-4653-823c-ecaf0533c02c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:20.095668 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9e6d6-1e0c-4653-823c-ecaf0533c02c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:20.095951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9e6d6-1e0c-4653-823c-ecaf0533c02c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:20.097150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9e6d6-1e0c-4653-823c-ecaf0533c02c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:20.098621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9e6d6-1e0c-4653-823c-ecaf0533c02c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:20.101966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 97/191] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:20.101966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9e6d6-1e0c-4653-823c-ecaf0533c02c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:20.147741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdca942-e4ea-46bf-81cd-85e67ea6173a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:20.149253 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdca942-e4ea-46bf-81cd-85e67ea6173a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:20.206852 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9e6d6-1e0c-4653-823c-ecaf0533c02c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:20.209190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:20.248003 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f1259-ab1d-4624-93fc-26d4f14e6a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:20.248996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f1259-ab1d-4624-93fc-26d4f14e6a3f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:20.249287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f1259-ab1d-4624-93fc-26d4f14e6a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:20.249572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f1259-ab1d-4624-93fc-26d4f14e6a3f None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:20.250343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f1259-ab1d-4624-93fc-26d4f14e6a3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:20.250653 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f1259-ab1d-4624-93fc-26d4f14e6a3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:20.252554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f1259-ab1d-4624-93fc-26d4f14e6a3f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:20.264467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 98/192] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:20.266204 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e90867-132b-472a-bd01-0a9d8001f11d None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:20.266562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e90867-132b-472a-bd01-0a9d8001f11d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:20.266921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e90867-132b-472a-bd01-0a9d8001f11d None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:20.268276 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 99/193] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:20.269108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e90867-132b-472a-bd01-0a9d8001f11d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:20.276421 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 95/194] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:20.277295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdca942-e4ea-46bf-81cd-85e67ea6173a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:20.300412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:20.393302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e90867-132b-472a-bd01-0a9d8001f11d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:20.401510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdca942-e4ea-46bf-81cd-85e67ea6173a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:20.433311 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe11594-efc4-4d0e-b740-05a05a349fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:20.433990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe11594-efc4-4d0e-b740-05a05a349fdc None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:20.434282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe11594-efc4-4d0e-b740-05a05a349fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:20.434565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe11594-efc4-4d0e-b740-05a05a349fdc None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:20.435279 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe11594-efc4-4d0e-b740-05a05a349fdc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:20.436810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe11594-efc4-4d0e-b740-05a05a349fdc None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:20.440738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eede65f-ddfd-4c5c-acb5-d9dda53391b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:20.441697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eede65f-ddfd-4c5c-acb5-d9dda53391b7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:20.442053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eede65f-ddfd-4c5c-acb5-d9dda53391b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:20.442407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eede65f-ddfd-4c5c-acb5-d9dda53391b7 None admin] PATH_INFO: `/v3/users/2ed9c364f8104907b156966f55ae4520` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:20.444278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eede65f-ddfd-4c5c-acb5-d9dda53391b7 None admin] RBAC: Authorizing `identity:get_user(user_id=2ed9c364f8104907b156966f55ae4520)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:20.445858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eede65f-ddfd-4c5c-acb5-d9dda53391b7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:20.447807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 96/195] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:20 2022] GET /identity/v3/users/2ed9c364f8104907b156966f55ae4520 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:20.448941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a60887-7c57-4298-8fff-ed19807e0a61 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:20.449296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a60887-7c57-4298-8fff-ed19807e0a61 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:20.449593 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a60887-7c57-4298-8fff-ed19807e0a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:20.465752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 100/196] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:53:20 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:53:20.465752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe11594-efc4-4d0e-b740-05a05a349fdc None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:20.476737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-93a60887-7c57-4298-8fff-ed19807e0a61 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:20.476737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-93a60887-7c57-4298-8fff-ed19807e0a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:20.496979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:20.587412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 97/197] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:20.587412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a60887-7c57-4298-8fff-ed19807e0a61 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:20.605437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:20.616028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe11594-efc4-4d0e-b740-05a05a349fdc None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:20.672025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f9ec8-998e-4e4c-a384-906336f0bf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:20.673932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f9ec8-998e-4e4c-a384-906336f0bf20 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:20.673932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f9ec8-998e-4e4c-a384-906336f0bf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:20.673932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f9ec8-998e-4e4c-a384-906336f0bf20 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:20.675350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f9ec8-998e-4e4c-a384-906336f0bf20 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:20.677163 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f9ec8-998e-4e4c-a384-906336f0bf20 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:20.680563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 101/198] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:20 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:20.681390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f9ec8-998e-4e4c-a384-906336f0bf20 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:20.719536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a60887-7c57-4298-8fff-ed19807e0a61 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:20.765770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e3fda-2834-47ea-b047-a616e4792815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:20.766438 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e3fda-2834-47ea-b047-a616e4792815 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:20.766722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e3fda-2834-47ea-b047-a616e4792815 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:20.766998 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e3fda-2834-47ea-b047-a616e4792815 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:20.808085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:20.823047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f9ec8-998e-4e4c-a384-906336f0bf20 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:20.856158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e3fda-2834-47ea-b047-a616e4792815 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:20.858214 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e3fda-2834-47ea-b047-a616e4792815 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:20.868840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ccd8b-7c65-432f-a636-826282de0709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:20.869769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ccd8b-7c65-432f-a636-826282de0709 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:20.870114 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ccd8b-7c65-432f-a636-826282de0709 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:20.870460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ccd8b-7c65-432f-a636-826282de0709 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:20.881876 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:20.885036 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ccd8b-7c65-432f-a636-826282de0709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f80ccd8b-7c65-432f-a636-826282de0709 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:20.893106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:20.894006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:20.894789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 102/199] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:20.895482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ccd8b-7c65-432f-a636-826282de0709 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:20.994121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 98/200] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:20.994912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e3fda-2834-47ea-b047-a616e4792815 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:21.027126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ccd8b-7c65-432f-a636-826282de0709 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:21.072021 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1fc0b-2150-4a56-b8ba-2517bfd2383e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:21.072951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1fc0b-2150-4a56-b8ba-2517bfd2383e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:21.074134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1fc0b-2150-4a56-b8ba-2517bfd2383e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:21.074418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1fc0b-2150-4a56-b8ba-2517bfd2383e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:21.075372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1fc0b-2150-4a56-b8ba-2517bfd2383e None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6, user.id=2ed9c364f8104907b156966f55ae4520) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:21.075853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1fc0b-2150-4a56-b8ba-2517bfd2383e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:21.077336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1fc0b-2150-4a56-b8ba-2517bfd2383e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:21.086577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 103/201] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:20 2022] GET /identity/v3/role_assignments?user.id=2ed9c364f8104907b156966f55ae4520&scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:21.087453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1fc0b-2150-4a56-b8ba-2517bfd2383e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:21.113545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e3fda-2834-47ea-b047-a616e4792815 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:21.128140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:21.149425 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:21.177274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360aacbb-9e1a-4c11-803a-9c5d04739e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:21.178442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360aacbb-9e1a-4c11-803a-9c5d04739e62 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:21.178971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360aacbb-9e1a-4c11-803a-9c5d04739e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:21.179535 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360aacbb-9e1a-4c11-803a-9c5d04739e62 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:21.180539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360aacbb-9e1a-4c11-803a-9c5d04739e62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:21.181066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360aacbb-9e1a-4c11-803a-9c5d04739e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:21.183091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360aacbb-9e1a-4c11-803a-9c5d04739e62 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:21.200045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 99/202] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:21.211445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360aacbb-9e1a-4c11-803a-9c5d04739e62 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:21.299140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1fc0b-2150-4a56-b8ba-2517bfd2383e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:21.365799 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360aacbb-9e1a-4c11-803a-9c5d04739e62 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:21.372177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59879cf-9a84-41e1-8794-6f46c55045e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:21.373068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59879cf-9a84-41e1-8794-6f46c55045e1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:21.373354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59879cf-9a84-41e1-8794-6f46c55045e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:21.373650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59879cf-9a84-41e1-8794-6f46c55045e1 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:21.375163 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59879cf-9a84-41e1-8794-6f46c55045e1 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:21.376746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59879cf-9a84-41e1-8794-6f46c55045e1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:21.378794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 104/203] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:21 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:21.384388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59879cf-9a84-41e1-8794-6f46c55045e1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:21.404826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a4d09-ab4e-4f33-8034-83594f75c2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:21.405707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a4d09-ab4e-4f33-8034-83594f75c2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:21.405986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a4d09-ab4e-4f33-8034-83594f75c2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:21.406260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a4d09-ab4e-4f33-8034-83594f75c2e9 None admin] PATH_INFO: `/v3/users/99c293e528654716abf3be17db345acb` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:21.407508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a4d09-ab4e-4f33-8034-83594f75c2e9 None admin] RBAC: Authorizing `identity:get_user(user_id=99c293e528654716abf3be17db345acb)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:21.408970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a4d09-ab4e-4f33-8034-83594f75c2e9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:21.410617 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 100/204] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:21 2022] GET /identity/v3/users/99c293e528654716abf3be17db345acb => generated 289 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:21.420041 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a4d09-ab4e-4f33-8034-83594f75c2e9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:21.503480 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:21.527778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59879cf-9a84-41e1-8794-6f46c55045e1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:21.545432 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:21.572598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec5651-c14e-44b1-8ace-ab89d604068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:21.573423 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec5651-c14e-44b1-8ace-ab89d604068e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:21.573739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec5651-c14e-44b1-8ace-ab89d604068e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:21.574014 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec5651-c14e-44b1-8ace-ab89d604068e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:21.575137 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec5651-c14e-44b1-8ace-ab89d604068e None admin] RBAC: Adding query filter params (role.id=ff3de46e4867421f990563b41a305d4d, scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6, user.id=2ed9c364f8104907b156966f55ae4520) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:21.575473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec5651-c14e-44b1-8ace-ab89d604068e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:21.577678 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec5651-c14e-44b1-8ace-ab89d604068e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:21.583789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 105/205] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:21 2022] GET /identity/v3/role_assignments?user.id=2ed9c364f8104907b156966f55ae4520&scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6&role.id=ff3de46e4867421f990563b41a305d4d => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:21.667345 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a4d09-ab4e-4f33-8034-83594f75c2e9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:21.710994 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4a451-88a5-4d1d-8118-3e63a4b3bbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:21.711694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4a451-88a5-4d1d-8118-3e63a4b3bbdd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:21.711983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4a451-88a5-4d1d-8118-3e63a4b3bbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:21.712264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4a451-88a5-4d1d-8118-3e63a4b3bbdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:21.787958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4a451-88a5-4d1d-8118-3e63a4b3bbdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:21.789509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4a451-88a5-4d1d-8118-3e63a4b3bbdd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:21.849071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:21.924598 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 101/206] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:21.930489 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec5651-c14e-44b1-8ace-ab89d604068e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:22.097293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec5651-c14e-44b1-8ace-ab89d604068e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:22.133660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189859fb-732a-4866-aca6-66f2caff0c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:22.134467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189859fb-732a-4866-aca6-66f2caff0c70 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:22.134754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189859fb-732a-4866-aca6-66f2caff0c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:22.135032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189859fb-732a-4866-aca6-66f2caff0c70 None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:22.136272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189859fb-732a-4866-aca6-66f2caff0c70 None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:22.137730 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189859fb-732a-4866-aca6-66f2caff0c70 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:22.139953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 106/207] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:21 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:22.148503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4a451-88a5-4d1d-8118-3e63a4b3bbdd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:22.307861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4a451-88a5-4d1d-8118-3e63a4b3bbdd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:22.355538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808c802-33c6-4f8a-9b99-e98fd7ba5d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:22.356223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808c802-33c6-4f8a-9b99-e98fd7ba5d60 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:22.356512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808c802-33c6-4f8a-9b99-e98fd7ba5d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:22.356793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808c802-33c6-4f8a-9b99-e98fd7ba5d60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:22.357646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808c802-33c6-4f8a-9b99-e98fd7ba5d60 None admin] RBAC: Adding query filter params (role.id=b628b5b418f143f7a1d72f29749e04fd, scope.project.id=cc744332e3b04129956875355de3f205, user.id=99c293e528654716abf3be17db345acb) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:22.357960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808c802-33c6-4f8a-9b99-e98fd7ba5d60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:22.363449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808c802-33c6-4f8a-9b99-e98fd7ba5d60 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:22.370789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 102/208] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:22 2022] GET /identity/v3/role_assignments?user.id=99c293e528654716abf3be17db345acb&scope.project.id=cc744332e3b04129956875355de3f205&role.id=b628b5b418f143f7a1d72f29749e04fd => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:22.926496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cf8a7-92ac-41ea-993e-d717eae1133c None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:22.926889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cf8a7-92ac-41ea-993e-d717eae1133c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:22.927065 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cf8a7-92ac-41ea-993e-d717eae1133c None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:22.929068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 107/209] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:22.936489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c638680-aa75-45fd-b454-ef5b2248b7fe None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:22.936966 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c638680-aa75-45fd-b454-ef5b2248b7fe None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:22.937052 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c638680-aa75-45fd-b454-ef5b2248b7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:22.962232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5c638680-aa75-45fd-b454-ef5b2248b7fe None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:22.963469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5c638680-aa75-45fd-b454-ef5b2248b7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:22.971794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:23.046747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 103/210] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:23.070344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cf8a7-92ac-41ea-993e-d717eae1133c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:23.087030 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:23.127256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a15353-87a0-4907-8c6c-7909de9f68e3 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:23.127556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a15353-87a0-4907-8c6c-7909de9f68e3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:23.128411 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a15353-87a0-4907-8c6c-7909de9f68e3 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:23.130231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 104/211] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:23.139177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a963a7-f4a3-4747-81ce-17f192871605 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:23.139849 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a963a7-f4a3-4747-81ce-17f192871605 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:23.140338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a963a7-f4a3-4747-81ce-17f192871605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:23.159819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b4a963a7-f4a3-4747-81ce-17f192871605 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:23.161829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b4a963a7-f4a3-4747-81ce-17f192871605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:23.189063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cf8a7-92ac-41ea-993e-d717eae1133c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:23.227759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b790553-91e4-4cf3-88cc-3553fc1019aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:23.228979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b790553-91e4-4cf3-88cc-3553fc1019aa None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:23.229469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b790553-91e4-4cf3-88cc-3553fc1019aa None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:23.229982 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b790553-91e4-4cf3-88cc-3553fc1019aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:23.241777 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b790553-91e4-4cf3-88cc-3553fc1019aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-7b790553-91e4-4cf3-88cc-3553fc1019aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:23.250303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:23.251169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:23.252291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:23.252291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:23.252291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:23.252291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:23.252291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:23.252291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:23.252291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:23.252291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:23.252291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:23.252291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:23.252291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:23.252291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:23.252902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 105/212] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:23.254793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 108/213] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:23.261304 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a963a7-f4a3-4747-81ce-17f192871605 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:23.281306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b790553-91e4-4cf3-88cc-3553fc1019aa None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:23.337418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:23.372061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a963a7-f4a3-4747-81ce-17f192871605 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:23.378892 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:23.418424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b790553-91e4-4cf3-88cc-3553fc1019aa None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:23.420980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f079e-bf0d-4e5a-bf42-0a87e9e45e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:23.422672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f079e-bf0d-4e5a-bf42-0a87e9e45e3d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:23.423029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f079e-bf0d-4e5a-bf42-0a87e9e45e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:23.423398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f079e-bf0d-4e5a-bf42-0a87e9e45e3d None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:23.424233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f079e-bf0d-4e5a-bf42-0a87e9e45e3d None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:23.424616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f079e-bf0d-4e5a-bf42-0a87e9e45e3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:23.426732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f079e-bf0d-4e5a-bf42-0a87e9e45e3d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:23.441620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 106/214] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:23.447047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f079e-bf0d-4e5a-bf42-0a87e9e45e3d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:23.466689 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb2d44-aa24-456d-afff-a2c974c586b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:23.467533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb2d44-aa24-456d-afff-a2c974c586b1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:23.467839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb2d44-aa24-456d-afff-a2c974c586b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:23.468123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb2d44-aa24-456d-afff-a2c974c586b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:23.469363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb2d44-aa24-456d-afff-a2c974c586b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:23.470965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb2d44-aa24-456d-afff-a2c974c586b1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:23.479419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 109/215] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:23.482011 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb2d44-aa24-456d-afff-a2c974c586b1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:23.605380 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f079e-bf0d-4e5a-bf42-0a87e9e45e3d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:23.613906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb2d44-aa24-456d-afff-a2c974c586b1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:23.641489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5dd47-efd9-4a8d-b3f0-577cf769aabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:23.642379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5dd47-efd9-4a8d-b3f0-577cf769aabf None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:23.642890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5dd47-efd9-4a8d-b3f0-577cf769aabf None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:23.642999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5dd47-efd9-4a8d-b3f0-577cf769aabf None admin] PATH_INFO: `/v3/users/2dd1dc9757a543b6abf99914d527824c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:23.645205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:23.649231 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04448f-1ec8-4811-92ef-9dc8fe61fff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:23.649946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04448f-1ec8-4811-92ef-9dc8fe61fff4 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:23.650236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04448f-1ec8-4811-92ef-9dc8fe61fff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:23.650558 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04448f-1ec8-4811-92ef-9dc8fe61fff4 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:23.651291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04448f-1ec8-4811-92ef-9dc8fe61fff4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:23.652790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04448f-1ec8-4811-92ef-9dc8fe61fff4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:23.661768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5dd47-efd9-4a8d-b3f0-577cf769aabf None admin] RBAC: Authorizing `identity:get_user(user_id=2dd1dc9757a543b6abf99914d527824c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:23.663701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5dd47-efd9-4a8d-b3f0-577cf769aabf None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:23.664316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:23.665553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 107/216] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:23 2022] GET /identity/v3/users/2dd1dc9757a543b6abf99914d527824c => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:23.668842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38eddb-ac28-4204-8ff6-b3fb505178dd None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:23.669187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38eddb-ac28-4204-8ff6-b3fb505178dd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:23.669399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38eddb-ac28-4204-8ff6-b3fb505178dd None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:23.670683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 108/217] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:23.677122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38eddb-ac28-4204-8ff6-b3fb505178dd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:23.687682 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 110/218] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:53:23 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:53:23.688858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65678caf-fc6f-4c2e-913e-130fe47c1439 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:23.689151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65678caf-fc6f-4c2e-913e-130fe47c1439 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:23.689467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65678caf-fc6f-4c2e-913e-130fe47c1439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:23.712028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-65678caf-fc6f-4c2e-913e-130fe47c1439 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:23.713194 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-65678caf-fc6f-4c2e-913e-130fe47c1439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:23.808657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 111/219] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:23.824044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38eddb-ac28-4204-8ff6-b3fb505178dd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:23.835044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65678caf-fc6f-4c2e-913e-130fe47c1439 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:23.866106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ce1f2-e1e4-4a75-bb66-fb30ed86f3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:23.866785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ce1f2-e1e4-4a75-bb66-fb30ed86f3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:23.867068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ce1f2-e1e4-4a75-bb66-fb30ed86f3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:23.867349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ce1f2-e1e4-4a75-bb66-fb30ed86f3d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:23.953928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:23.969529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ce1f2-e1e4-4a75-bb66-fb30ed86f3d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:23.971033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ce1f2-e1e4-4a75-bb66-fb30ed86f3d4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:23.993382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65678caf-fc6f-4c2e-913e-130fe47c1439 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:24.029691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:24.043966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5df5f-4928-4247-99ab-537356fbad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:24.044774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5df5f-4928-4247-99ab-537356fbad34 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:24.045061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5df5f-4928-4247-99ab-537356fbad34 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:24.045336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5df5f-4928-4247-99ab-537356fbad34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:24.056838 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5df5f-4928-4247-99ab-537356fbad34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-b2f5df5f-4928-4247-99ab-537356fbad34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:24.064781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:24.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:24.066448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 112/220] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:53:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:24.071673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa03a91-e27e-4c52-a973-b9e668981275 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:24.072239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa03a91-e27e-4c52-a973-b9e668981275 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:24.072239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa03a91-e27e-4c52-a973-b9e668981275 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:24.073560 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 113/221] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:24.075796 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa03a91-e27e-4c52-a973-b9e668981275 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:24.108572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 109/222] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:24.110213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449557f7-b545-480b-8b27-d5d5aa1b4a79 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:24.110732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449557f7-b545-480b-8b27-d5d5aa1b4a79 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:24.111224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449557f7-b545-480b-8b27-d5d5aa1b4a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:24.128989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-449557f7-b545-480b-8b27-d5d5aa1b4a79 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:24.130231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-449557f7-b545-480b-8b27-d5d5aa1b4a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:24.222630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 110/223] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:24.229260 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa03a91-e27e-4c52-a973-b9e668981275 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:24.230493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449557f7-b545-480b-8b27-d5d5aa1b4a79 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:24.271781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa446ab4-bf8d-4b70-acbb-f9bedfa75bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:24.272464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa446ab4-bf8d-4b70-acbb-f9bedfa75bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:24.272762 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa446ab4-bf8d-4b70-acbb-f9bedfa75bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:24.273170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa446ab4-bf8d-4b70-acbb-f9bedfa75bf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:24.274072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa446ab4-bf8d-4b70-acbb-f9bedfa75bf1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:24.274380 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa446ab4-bf8d-4b70-acbb-f9bedfa75bf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:24.275534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:24.277513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa446ab4-bf8d-4b70-acbb-f9bedfa75bf1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:24.290966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 114/224] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:24.296718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa446ab4-bf8d-4b70-acbb-f9bedfa75bf1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:24.359911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:24.380024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449557f7-b545-480b-8b27-d5d5aa1b4a79 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:24.419195 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa446ab4-bf8d-4b70-acbb-f9bedfa75bf1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:24.425972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467daf1-2a16-48f7-84aa-f61c54712274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:24.426841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467daf1-2a16-48f7-84aa-f61c54712274 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:24.427122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467daf1-2a16-48f7-84aa-f61c54712274 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:24.427408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467daf1-2a16-48f7-84aa-f61c54712274 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:24.429656 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467daf1-2a16-48f7-84aa-f61c54712274 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:24.431655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467daf1-2a16-48f7-84aa-f61c54712274 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:24.434560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 111/225] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:24 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:24.435178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467daf1-2a16-48f7-84aa-f61c54712274 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:24.472509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cb74e-8f82-4393-a9ab-70e0e26e4c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:24.473419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cb74e-8f82-4393-a9ab-70e0e26e4c81 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:24.473698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cb74e-8f82-4393-a9ab-70e0e26e4c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:24.473937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cb74e-8f82-4393-a9ab-70e0e26e4c81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:24.489884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cb74e-8f82-4393-a9ab-70e0e26e4c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-8a8cb74e-8f82-4393-a9ab-70e0e26e4c81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:24.499830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:24.501495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 115/226] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:24.502928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cb74e-8f82-4393-a9ab-70e0e26e4c81 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:24.573181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:24.584621 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467daf1-2a16-48f7-84aa-f61c54712274 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:24.622407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751f31b-4cdb-4187-bc95-6e441fd45f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:24.623708 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751f31b-4cdb-4187-bc95-6e441fd45f56 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:24.624200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751f31b-4cdb-4187-bc95-6e441fd45f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:24.624681 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751f31b-4cdb-4187-bc95-6e441fd45f56 None admin] PATH_INFO: `/v3/users/2ed9c364f8104907b156966f55ae4520` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:24.626103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751f31b-4cdb-4187-bc95-6e441fd45f56 None admin] RBAC: Authorizing `identity:get_user(user_id=2ed9c364f8104907b156966f55ae4520)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:24.627782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751f31b-4cdb-4187-bc95-6e441fd45f56 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:24.629661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 112/227] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:24 2022] GET /identity/v3/users/2ed9c364f8104907b156966f55ae4520 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:24.630696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751f31b-4cdb-4187-bc95-6e441fd45f56 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:24.636489 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cb74e-8f82-4393-a9ab-70e0e26e4c81 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:24.648806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:24.684708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883cc50-7134-4d05-a8c9-98d1025210dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:24.685644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883cc50-7134-4d05-a8c9-98d1025210dd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:24.685938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883cc50-7134-4d05-a8c9-98d1025210dd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:24.686223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883cc50-7134-4d05-a8c9-98d1025210dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:24.687073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883cc50-7134-4d05-a8c9-98d1025210dd None admin] RBAC: Adding query filter params (role.id=9a562f9a0cad4304852f3183eef398d2, scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6, user.id=2dd1dc9757a543b6abf99914d527824c) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:24.687414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883cc50-7134-4d05-a8c9-98d1025210dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:24.689652 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883cc50-7134-4d05-a8c9-98d1025210dd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:24.699459 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 116/228] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:24 2022] GET /identity/v3/role_assignments?user.id=2dd1dc9757a543b6abf99914d527824c&scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6&role.id=9a562f9a0cad4304852f3183eef398d2 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:24.700603 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883cc50-7134-4d05-a8c9-98d1025210dd None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:24.796124 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751f31b-4cdb-4187-bc95-6e441fd45f56 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:24.873098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30c93d-c9e7-4df6-8f60-07e7e3061d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:24.874376 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30c93d-c9e7-4df6-8f60-07e7e3061d91 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:24.876034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30c93d-c9e7-4df6-8f60-07e7e3061d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:24.877430 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30c93d-c9e7-4df6-8f60-07e7e3061d91 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:24.878641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30c93d-c9e7-4df6-8f60-07e7e3061d91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:24.880503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30c93d-c9e7-4df6-8f60-07e7e3061d91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:24.882244 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30c93d-c9e7-4df6-8f60-07e7e3061d91 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:24.901164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 113/229] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:24.908066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30c93d-c9e7-4df6-8f60-07e7e3061d91 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:24.922385 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883cc50-7134-4d05-a8c9-98d1025210dd None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:24.944169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:24.993587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea216408-f4eb-4ac4-a33e-4eff96d36df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:24.995035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea216408-f4eb-4ac4-a33e-4eff96d36df9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:24.996836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea216408-f4eb-4ac4-a33e-4eff96d36df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:24.998161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea216408-f4eb-4ac4-a33e-4eff96d36df9 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:25.001201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea216408-f4eb-4ac4-a33e-4eff96d36df9 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:25.003045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea216408-f4eb-4ac4-a33e-4eff96d36df9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:25.008973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 117/230] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:24 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:25.015048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea216408-f4eb-4ac4-a33e-4eff96d36df9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:25.081707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:25.098641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30c93d-c9e7-4df6-8f60-07e7e3061d91 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:25.175593 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db1bcd-eb04-4e5a-8811-9a6464557d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:25.178203 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db1bcd-eb04-4e5a-8811-9a6464557d28 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:25.179086 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db1bcd-eb04-4e5a-8811-9a6464557d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:25.180483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db1bcd-eb04-4e5a-8811-9a6464557d28 None admin] PATH_INFO: `/v3/users/2ed9c364f8104907b156966f55ae4520` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:25.183642 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db1bcd-eb04-4e5a-8811-9a6464557d28 None admin] RBAC: Authorizing `identity:get_user(user_id=2ed9c364f8104907b156966f55ae4520)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:25.185701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db1bcd-eb04-4e5a-8811-9a6464557d28 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:25.189092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 114/231] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:24 2022] GET /identity/v3/users/2ed9c364f8104907b156966f55ae4520 => generated 283 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:25.196988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db1bcd-eb04-4e5a-8811-9a6464557d28 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:25.221030 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea216408-f4eb-4ac4-a33e-4eff96d36df9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:25.282611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091755f2-0226-4d5d-9d56-cbca24e840b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:25.283634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091755f2-0226-4d5d-9d56-cbca24e840b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:25.283917 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091755f2-0226-4d5d-9d56-cbca24e840b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:25.284358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091755f2-0226-4d5d-9d56-cbca24e840b6 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6/users/2ed9c364f8104907b156966f55ae4520/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:25.286290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091755f2-0226-4d5d-9d56-cbca24e840b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=431ab0f9d63c471fb738c37e4c9cbcb6, user_id=2ed9c364f8104907b156966f55ae4520, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:25.287887 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091755f2-0226-4d5d-9d56-cbca24e840b6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:25.319416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 118/232] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:53:25 2022] PUT /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6/users/2ed9c364f8104907b156966f55ae4520/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:25.361906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:25.382351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db1bcd-eb04-4e5a-8811-9a6464557d28 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:25.435347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ad97f-fbb9-49a3-ac93-08e0bf84e1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:25.436663 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ad97f-fbb9-49a3-ac93-08e0bf84e1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:25.437182 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ad97f-fbb9-49a3-ac93-08e0bf84e1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:25.437659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ad97f-fbb9-49a3-ac93-08e0bf84e1b6 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:25.439112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ad97f-fbb9-49a3-ac93-08e0bf84e1b6 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:25.440845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ad97f-fbb9-49a3-ac93-08e0bf84e1b6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:25.442856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 115/233] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:25 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:25.449244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091755f2-0226-4d5d-9d56-cbca24e840b6 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:25.569564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:25.573169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091755f2-0226-4d5d-9d56-cbca24e840b6 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:25.614250 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d6fc4-e59f-424c-a10f-eb968a5584e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:25.616120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d6fc4-e59f-424c-a10f-eb968a5584e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:25.616451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d6fc4-e59f-424c-a10f-eb968a5584e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:25.616737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d6fc4-e59f-424c-a10f-eb968a5584e3 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6/users/2ed9c364f8104907b156966f55ae4520/roles/ff3de46e4867421f990563b41a305d4d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:25.619098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d6fc4-e59f-424c-a10f-eb968a5584e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=431ab0f9d63c471fb738c37e4c9cbcb6, user_id=2ed9c364f8104907b156966f55ae4520, role_id=ff3de46e4867421f990563b41a305d4d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:25.620599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d6fc4-e59f-424c-a10f-eb968a5584e3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:25.639215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 119/234] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:53:25 2022] PUT /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6/users/2ed9c364f8104907b156966f55ae4520/roles/ff3de46e4867421f990563b41a305d4d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:26.061425 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46183f-9201-403a-ad43-e18cc989eb9e None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:26.061818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46183f-9201-403a-ad43-e18cc989eb9e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:26.061992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46183f-9201-403a-ad43-e18cc989eb9e None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:26.063680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 116/235] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:26.068922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d74a6c-9a10-4995-9509-028a36131ee4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:26.069211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d74a6c-9a10-4995-9509-028a36131ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:26.069490 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d74a6c-9a10-4995-9509-028a36131ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:26.092398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a6d74a6c-9a10-4995-9509-028a36131ee4 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:26.093585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a6d74a6c-9a10-4995-9509-028a36131ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:26.172135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b40d76-111a-4e2f-9e8a-e4a6d61c37fa None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:26.172690 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b40d76-111a-4e2f-9e8a-e4a6d61c37fa None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:26.172690 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b40d76-111a-4e2f-9e8a-e4a6d61c37fa None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:26.174422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 117/236] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:26.180671 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb04e7d-0395-42d8-b3d4-f3d1e632f30f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:26.180966 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb04e7d-0395-42d8-b3d4-f3d1e632f30f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:26.181292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb04e7d-0395-42d8-b3d4-f3d1e632f30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:26.203877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 120/237] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:26.211310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-bdb04e7d-0395-42d8-b3d4-f3d1e632f30f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:26.212535 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bdb04e7d-0395-42d8-b3d4-f3d1e632f30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:26.248734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d74a6c-9a10-4995-9509-028a36131ee4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:26.341368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 118/238] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:26.380619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb04e7d-0395-42d8-b3d4-f3d1e632f30f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:26.388113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d74a6c-9a10-4995-9509-028a36131ee4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:26.390647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:26.429650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcaccc-29dc-400a-ae43-d795485c3864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:26.431306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcaccc-29dc-400a-ae43-d795485c3864 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:26.431938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcaccc-29dc-400a-ae43-d795485c3864 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:26.432679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcaccc-29dc-400a-ae43-d795485c3864 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:26.444634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dcaccc-29dc-400a-ae43-d795485c3864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-15dcaccc-29dc-400a-ae43-d795485c3864 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:26.454013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:26.454878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:26.455724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:26.455724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:26.455724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:26.455724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:26.455724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:26.455724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:26.455724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:26.455724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:26.455724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:26.455724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:26.455724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:26.455724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:26.458120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 121/239] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:26.465997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcaccc-29dc-400a-ae43-d795485c3864 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:26.518322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:26.534361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb04e7d-0395-42d8-b3d4-f3d1e632f30f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:26.577441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb275793-930a-4afd-ba60-1027cd479232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:26.580427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb275793-930a-4afd-ba60-1027cd479232 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:26.580723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb275793-930a-4afd-ba60-1027cd479232 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:26.581002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb275793-930a-4afd-ba60-1027cd479232 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:26.582919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb275793-930a-4afd-ba60-1027cd479232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:26.584455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb275793-930a-4afd-ba60-1027cd479232 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:26.586227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 119/240] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:26.592503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb275793-930a-4afd-ba60-1027cd479232 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:26.681976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcaccc-29dc-400a-ae43-d795485c3864 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:26.705831 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb275793-930a-4afd-ba60-1027cd479232 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:26.744703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac2a10-4a80-4fd5-804b-975f85bc7346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:26.746569 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac2a10-4a80-4fd5-804b-975f85bc7346 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:26.746852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac2a10-4a80-4fd5-804b-975f85bc7346 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:26.747128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac2a10-4a80-4fd5-804b-975f85bc7346 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:26.747885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac2a10-4a80-4fd5-804b-975f85bc7346 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:26.749336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac2a10-4a80-4fd5-804b-975f85bc7346 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:26.764549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2cdbe-deb3-4162-a868-f9778d5cdb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:26.765231 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2cdbe-deb3-4162-a868-f9778d5cdb42 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:26.765512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2cdbe-deb3-4162-a868-f9778d5cdb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:26.765787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2cdbe-deb3-4162-a868-f9778d5cdb42 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:26.766535 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2cdbe-deb3-4162-a868-f9778d5cdb42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:26.766837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2cdbe-deb3-4162-a868-f9778d5cdb42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:26.768597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2cdbe-deb3-4162-a868-f9778d5cdb42 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:26.785261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 122/241] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:26.791074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2cdbe-deb3-4162-a868-f9778d5cdb42 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:26.812287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:26.813430 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 120/242] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:53:26 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:53:26.920119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2cdbe-deb3-4162-a868-f9778d5cdb42 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:26.958356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3091e88-d68d-49a9-ae9c-31f8ebe83e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:26.959290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3091e88-d68d-49a9-ae9c-31f8ebe83e7f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:26.959702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3091e88-d68d-49a9-ae9c-31f8ebe83e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:26.959989 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3091e88-d68d-49a9-ae9c-31f8ebe83e7f None admin] PATH_INFO: `/v3/users/cdd89fadb95241f68d641ef50f4c9021` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:26.976069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3091e88-d68d-49a9-ae9c-31f8ebe83e7f None admin] RBAC: Authorizing `identity:get_user(user_id=cdd89fadb95241f68d641ef50f4c9021)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:26.977561 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3091e88-d68d-49a9-ae9c-31f8ebe83e7f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:26.979308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 123/243] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:26 2022] GET /identity/v3/users/cdd89fadb95241f68d641ef50f4c9021 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:26.988468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac2a10-4a80-4fd5-804b-975f85bc7346 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:27.002351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:27.096292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac2a10-4a80-4fd5-804b-975f85bc7346 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:27.140712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca3c5e-e3e7-4894-a88c-fc25933f301d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:27.141384 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca3c5e-e3e7-4894-a88c-fc25933f301d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:27.141677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca3c5e-e3e7-4894-a88c-fc25933f301d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:27.141960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca3c5e-e3e7-4894-a88c-fc25933f301d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:27.226982 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca3c5e-e3e7-4894-a88c-fc25933f301d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:27.228534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca3c5e-e3e7-4894-a88c-fc25933f301d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:27.240436 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:27.302896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4c942-1c37-4ed3-b49b-35eeee77c33f None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:27.303468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4c942-1c37-4ed3-b49b-35eeee77c33f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:27.303468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4c942-1c37-4ed3-b49b-35eeee77c33f None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:27.305086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 124/244] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:27.310506 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725fc62-01ae-40b3-91b0-ae5699f859e0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:27.310789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725fc62-01ae-40b3-91b0-ae5699f859e0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:27.311066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725fc62-01ae-40b3-91b0-ae5699f859e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:27.345242 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8725fc62-01ae-40b3-91b0-ae5699f859e0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:27.346448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8725fc62-01ae-40b3-91b0-ae5699f859e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:27.348176 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 121/245] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:27.354207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca3c5e-e3e7-4894-a88c-fc25933f301d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:27.429232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:27.437385 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 125/246] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:27.461421 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725fc62-01ae-40b3-91b0-ae5699f859e0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:27.471347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca3c5e-e3e7-4894-a88c-fc25933f301d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:27.483130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:27.512024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc511b6-b992-482f-8187-123dee2e8f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:27.513288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc511b6-b992-482f-8187-123dee2e8f65 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:27.513773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc511b6-b992-482f-8187-123dee2e8f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:27.514250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc511b6-b992-482f-8187-123dee2e8f65 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:27.515755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc511b6-b992-482f-8187-123dee2e8f65 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:27.517438 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc511b6-b992-482f-8187-123dee2e8f65 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:27.519420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 122/247] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:27 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:27.526147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc511b6-b992-482f-8187-123dee2e8f65 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:27.577034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725fc62-01ae-40b3-91b0-ae5699f859e0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:27.619212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887114d-421d-459f-8a2d-b929bc1d85d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:27.620123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887114d-421d-459f-8a2d-b929bc1d85d4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:27.620405 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887114d-421d-459f-8a2d-b929bc1d85d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:27.620680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887114d-421d-459f-8a2d-b929bc1d85d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:27.638403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887114d-421d-459f-8a2d-b929bc1d85d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-0887114d-421d-459f-8a2d-b929bc1d85d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:27.651657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:27.652609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:27.653373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 126/248] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:53:27.661559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887114d-421d-459f-8a2d-b929bc1d85d4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:27.670168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc511b6-b992-482f-8187-123dee2e8f65 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:27.714409 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712d301-c133-4d16-8c37-212c5f096f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:27.715455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712d301-c133-4d16-8c37-212c5f096f77 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:27.715970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712d301-c133-4d16-8c37-212c5f096f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:27.716449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712d301-c133-4d16-8c37-212c5f096f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:27.717492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712d301-c133-4d16-8c37-212c5f096f77 None admin] RBAC: Adding query filter params (role.id=b628b5b418f143f7a1d72f29749e04fd, scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6, user.id=cdd89fadb95241f68d641ef50f4c9021) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:27.718005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712d301-c133-4d16-8c37-212c5f096f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:27.719716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712d301-c133-4d16-8c37-212c5f096f77 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:27.735513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 123/249] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:27 2022] GET /identity/v3/role_assignments?user.id=cdd89fadb95241f68d641ef50f4c9021&scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6&role.id=b628b5b418f143f7a1d72f29749e04fd => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:27.739407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:27.769886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616c8c6-7e7c-4071-b6fd-1ccde210835f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:27.770547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616c8c6-7e7c-4071-b6fd-1ccde210835f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:27.771032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616c8c6-7e7c-4071-b6fd-1ccde210835f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:27.773622 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 124/250] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:27.779646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35666eb3-4265-418f-82ae-7c014ab2c59d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:27.780300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35666eb3-4265-418f-82ae-7c014ab2c59d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:27.780880 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35666eb3-4265-418f-82ae-7c014ab2c59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:27.791104 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887114d-421d-459f-8a2d-b929bc1d85d4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:27.800680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-35666eb3-4265-418f-82ae-7c014ab2c59d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:27.802442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-35666eb3-4265-418f-82ae-7c014ab2c59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:27.839906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2fcb5-2065-449f-9068-8ee94165a253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:27.840594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2fcb5-2065-449f-9068-8ee94165a253 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:27.840909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2fcb5-2065-449f-9068-8ee94165a253 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:27.841431 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2fcb5-2065-449f-9068-8ee94165a253 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:27.842256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2fcb5-2065-449f-9068-8ee94165a253 None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:27.842565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2fcb5-2065-449f-9068-8ee94165a253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:27.844083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2fcb5-2065-449f-9068-8ee94165a253 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:27.848668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:27.857812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 127/251] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:27.863407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2fcb5-2065-449f-9068-8ee94165a253 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:27.889490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 125/252] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:27.915051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35666eb3-4265-418f-82ae-7c014ab2c59d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:27.999628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2fcb5-2065-449f-9068-8ee94165a253 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:28.034686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:28.036869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35666eb3-4265-418f-82ae-7c014ab2c59d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:28.051092 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de986e-4803-4068-8667-541b45de2d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:28.055113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de986e-4803-4068-8667-541b45de2d0a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:28.057074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de986e-4803-4068-8667-541b45de2d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:28.058536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de986e-4803-4068-8667-541b45de2d0a None admin] PATH_INFO: `/v3/users/2dd1dc9757a543b6abf99914d527824c` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:28.061727 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de986e-4803-4068-8667-541b45de2d0a None admin] RBAC: Authorizing `identity:get_user(user_id=2dd1dc9757a543b6abf99914d527824c)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:28.064369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de986e-4803-4068-8667-541b45de2d0a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:28.068303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 128/253] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:27 2022] GET /identity/v3/users/2dd1dc9757a543b6abf99914d527824c => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:28.075309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de986e-4803-4068-8667-541b45de2d0a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:28.108813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc610f-39f2-4d5c-88b1-d710a305d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:28.110005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc610f-39f2-4d5c-88b1-d710a305d8fa None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:28.110518 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc610f-39f2-4d5c-88b1-d710a305d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:28.110998 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc610f-39f2-4d5c-88b1-d710a305d8fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:28.129092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc610f-39f2-4d5c-88b1-d710a305d8fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6ecc610f-39f2-4d5c-88b1-d710a305d8fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:28.138324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:28.139272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:28.140123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:28.140123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:28.140123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:28.140123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:28.140123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:28.140123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:28.140123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:28.140123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:28.140123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:28.140123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:28.140123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:28.140123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:28.149339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 126/254] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:53:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:53:28.150622 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc610f-39f2-4d5c-88b1-d710a305d8fa None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:28.219470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:28.299885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de986e-4803-4068-8667-541b45de2d0a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:28.344248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc610f-39f2-4d5c-88b1-d710a305d8fa None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:28.372821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1375e60-cb92-4ea6-beb9-825dc6cd2d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:28.374522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1375e60-cb92-4ea6-beb9-825dc6cd2d44 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:28.375012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1375e60-cb92-4ea6-beb9-825dc6cd2d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:28.375598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1375e60-cb92-4ea6-beb9-825dc6cd2d44 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:28.380231 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1375e60-cb92-4ea6-beb9-825dc6cd2d44 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:28.382601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1375e60-cb92-4ea6-beb9-825dc6cd2d44 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:28.384747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 129/255] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:28 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:28.390603 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1375e60-cb92-4ea6-beb9-825dc6cd2d44 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:28.391225 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecacea7-6462-4200-8358-50ca67298a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:28.392016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecacea7-6462-4200-8358-50ca67298a1e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:28.392307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecacea7-6462-4200-8358-50ca67298a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:28.392581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecacea7-6462-4200-8358-50ca67298a1e None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:28.393347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecacea7-6462-4200-8358-50ca67298a1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:28.393654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecacea7-6462-4200-8358-50ca67298a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:28.397077 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecacea7-6462-4200-8358-50ca67298a1e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:28.406104 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 127/256] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:28.412435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecacea7-6462-4200-8358-50ca67298a1e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:28.512530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:28.529855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:28.537315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecacea7-6462-4200-8358-50ca67298a1e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:28.563170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1375e60-cb92-4ea6-beb9-825dc6cd2d44 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:28.581123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c49ffa-9f2c-4cd9-8b9c-a2f035683cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:28.582607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c49ffa-9f2c-4cd9-8b9c-a2f035683cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:28.582995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c49ffa-9f2c-4cd9-8b9c-a2f035683cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:28.583373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c49ffa-9f2c-4cd9-8b9c-a2f035683cd0 None admin] PATH_INFO: `/v3/users/2ed9c364f8104907b156966f55ae4520` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:28.585336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c49ffa-9f2c-4cd9-8b9c-a2f035683cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=2ed9c364f8104907b156966f55ae4520)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:28.587407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c49ffa-9f2c-4cd9-8b9c-a2f035683cd0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:28.589540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 128/257] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:28 2022] GET /identity/v3/users/2ed9c364f8104907b156966f55ae4520 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:28.594991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c49ffa-9f2c-4cd9-8b9c-a2f035683cd0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:28.604547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89475f10-52ad-4410-853d-3acb3637aa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:28.605713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89475f10-52ad-4410-853d-3acb3637aa85 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:28.606181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89475f10-52ad-4410-853d-3acb3637aa85 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:28.606683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89475f10-52ad-4410-853d-3acb3637aa85 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6/users/2dd1dc9757a543b6abf99914d527824c/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:28.609067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89475f10-52ad-4410-853d-3acb3637aa85 None admin] RBAC: Authorizing `identity:create_grant(project_id=431ab0f9d63c471fb738c37e4c9cbcb6, user_id=2dd1dc9757a543b6abf99914d527824c, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:28.610834 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89475f10-52ad-4410-853d-3acb3637aa85 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:28.627309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 130/258] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:53:28 2022] PUT /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6/users/2dd1dc9757a543b6abf99914d527824c/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:53:28.665419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7af13-09b1-4926-9091-d071dfeba1b7 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:28.666007 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7af13-09b1-4926-9091-d071dfeba1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:28.666543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7af13-09b1-4926-9091-d071dfeba1b7 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:28.668614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 131/259] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:53:28.679714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6986249-1dc9-4003-a83c-fe397ac4b57b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:28.680316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6986249-1dc9-4003-a83c-fe397ac4b57b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:28.680804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6986249-1dc9-4003-a83c-fe397ac4b57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:28.704754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a6986249-1dc9-4003-a83c-fe397ac4b57b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:28.706487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a6986249-1dc9-4003-a83c-fe397ac4b57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:28.785849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 132/260] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:28.811296 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6986249-1dc9-4003-a83c-fe397ac4b57b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:28.818082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c49ffa-9f2c-4cd9-8b9c-a2f035683cd0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:28.868787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:28.900921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aad010-1d27-402c-8c31-70b73a09df20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:28.901609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aad010-1d27-402c-8c31-70b73a09df20 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:28.901891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aad010-1d27-402c-8c31-70b73a09df20 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:28.902167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aad010-1d27-402c-8c31-70b73a09df20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:28.935136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6986249-1dc9-4003-a83c-fe397ac4b57b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:28.959548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:28.975889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa69c4-432c-4fad-9a96-df019acd034e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:28.977237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa69c4-432c-4fad-9a96-df019acd034e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:28.977723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa69c4-432c-4fad-9a96-df019acd034e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:28.978202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa69c4-432c-4fad-9a96-df019acd034e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:28.990048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa69c4-432c-4fad-9a96-df019acd034e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-d3fa69c4-432c-4fad-9a96-df019acd034e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:28.999161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:29.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:29.000939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:29.000939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:29.000939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:29.000939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:29.000939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:29.000939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:29.000939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:29.000939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:29.000939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:29.000939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:29.000939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:29.000939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:29.003456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 133/261] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:53:29.010187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa69c4-432c-4fad-9a96-df019acd034e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:29.038380 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aad010-1d27-402c-8c31-70b73a09df20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:29.039880 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aad010-1d27-402c-8c31-70b73a09df20 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:29.119077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa69c4-432c-4fad-9a96-df019acd034e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:29.129393 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:29.157581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32707a2f-8626-4f91-a08f-acae876fa9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:29.158268 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32707a2f-8626-4f91-a08f-acae876fa9ba None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:29.158563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32707a2f-8626-4f91-a08f-acae876fa9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:29.158848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32707a2f-8626-4f91-a08f-acae876fa9ba None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:29.159639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32707a2f-8626-4f91-a08f-acae876fa9ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:29.160279 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32707a2f-8626-4f91-a08f-acae876fa9ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:29.162079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32707a2f-8626-4f91-a08f-acae876fa9ba None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:29.171027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 134/262] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:29.176955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32707a2f-8626-4f91-a08f-acae876fa9ba None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:29.298698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32707a2f-8626-4f91-a08f-acae876fa9ba None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:29.322238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 129/263] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:53:29.328339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aad010-1d27-402c-8c31-70b73a09df20 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:29.340533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71967011-19d4-4e4c-9e94-e978a0a7ae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:29.341479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71967011-19d4-4e4c-9e94-e978a0a7ae2c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:29.341769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71967011-19d4-4e4c-9e94-e978a0a7ae2c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:29.342047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71967011-19d4-4e4c-9e94-e978a0a7ae2c None admin] PATH_INFO: `/v3/users/2ed9c364f8104907b156966f55ae4520` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:29.343318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71967011-19d4-4e4c-9e94-e978a0a7ae2c None admin] RBAC: Authorizing `identity:get_user(user_id=2ed9c364f8104907b156966f55ae4520)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:29.344818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71967011-19d4-4e4c-9e94-e978a0a7ae2c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:29.348606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 135/264] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:29 2022] GET /identity/v3/users/2ed9c364f8104907b156966f55ae4520 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:29.355411 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71967011-19d4-4e4c-9e94-e978a0a7ae2c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:29.431872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:29.458536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:29.538170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aad010-1d27-402c-8c31-70b73a09df20 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:29.563477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71967011-19d4-4e4c-9e94-e978a0a7ae2c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:29.581089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc18ea-d02d-4a58-af3f-4ead4d5fd3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:29.582379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc18ea-d02d-4a58-af3f-4ead4d5fd3ca None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:29.582670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc18ea-d02d-4a58-af3f-4ead4d5fd3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:29.582956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc18ea-d02d-4a58-af3f-4ead4d5fd3ca None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:29.584324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc18ea-d02d-4a58-af3f-4ead4d5fd3ca None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:29.586206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc18ea-d02d-4a58-af3f-4ead4d5fd3ca None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:29.588529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 130/265] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:29 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:29.594198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc18ea-d02d-4a58-af3f-4ead4d5fd3ca None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:29.653249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839c2be-0ef3-4a12-a323-f170663f7d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:29.654096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839c2be-0ef3-4a12-a323-f170663f7d26 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:29.654388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839c2be-0ef3-4a12-a323-f170663f7d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:29.654665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839c2be-0ef3-4a12-a323-f170663f7d26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:29.774610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc18ea-d02d-4a58-af3f-4ead4d5fd3ca None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:29.798250 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839c2be-0ef3-4a12-a323-f170663f7d26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:29.800193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839c2be-0ef3-4a12-a323-f170663f7d26 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:29.827710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:29.852272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df725b8e-1441-420f-99dc-6731655f41fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:29.853377 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df725b8e-1441-420f-99dc-6731655f41fb None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:29.853885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df725b8e-1441-420f-99dc-6731655f41fb None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:29.854380 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df725b8e-1441-420f-99dc-6731655f41fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:29.855489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df725b8e-1441-420f-99dc-6731655f41fb None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6, user.id=2ed9c364f8104907b156966f55ae4520) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:29.856022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df725b8e-1441-420f-99dc-6731655f41fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:29.857688 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df725b8e-1441-420f-99dc-6731655f41fb None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:29.870622 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 131/266] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:29 2022] GET /identity/v3/role_assignments?user.id=2ed9c364f8104907b156966f55ae4520&scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:29.892264 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:29.989161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 136/267] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:29.995259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df725b8e-1441-420f-99dc-6731655f41fb None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:30.144911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df725b8e-1441-420f-99dc-6731655f41fb None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:30.194236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898622f-0384-4071-95d4-3c2eef8672ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:30.195543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898622f-0384-4071-95d4-3c2eef8672ca None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:30.196027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898622f-0384-4071-95d4-3c2eef8672ca None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:30.196499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898622f-0384-4071-95d4-3c2eef8672ca None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:30.202636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898622f-0384-4071-95d4-3c2eef8672ca None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:30.204385 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898622f-0384-4071-95d4-3c2eef8672ca None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:30.206397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 132/268] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:29 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:30.216493 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839c2be-0ef3-4a12-a323-f170663f7d26 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:30.355778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839c2be-0ef3-4a12-a323-f170663f7d26 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:30.390634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c2841-376e-47c0-84cb-8d444511e070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:30.391293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c2841-376e-47c0-84cb-8d444511e070 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:30.391601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c2841-376e-47c0-84cb-8d444511e070 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:30.391877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c2841-376e-47c0-84cb-8d444511e070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:30.392707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c2841-376e-47c0-84cb-8d444511e070 None admin] RBAC: Adding query filter params (role.id=ff3de46e4867421f990563b41a305d4d, scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6, user.id=2ed9c364f8104907b156966f55ae4520) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:30.393016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c2841-376e-47c0-84cb-8d444511e070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:30.394478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c2841-376e-47c0-84cb-8d444511e070 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:30.400831 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 137/269] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:30 2022] GET /identity/v3/role_assignments?user.id=2ed9c364f8104907b156966f55ae4520&scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6&role.id=ff3de46e4867421f990563b41a305d4d => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:30.466629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f0e7d-ff6b-4197-baf4-b1b6fa310d0e None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:30.467624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f0e7d-ff6b-4197-baf4-b1b6fa310d0e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:30.468133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f0e7d-ff6b-4197-baf4-b1b6fa310d0e None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:30.469775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 133/270] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:30.476155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8d91a-5b2a-4aa4-9504-b506443f983d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:30.476811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8d91a-5b2a-4aa4-9504-b506443f983d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:30.476811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8d91a-5b2a-4aa4-9504-b506443f983d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:30.482851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:30.500651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-62b8d91a-5b2a-4aa4-9504-b506443f983d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:30.501905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-62b8d91a-5b2a-4aa4-9504-b506443f983d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:30.643659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 138/271] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:30.672909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f0e7d-ff6b-4197-baf4-b1b6fa310d0e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:30.761825 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:30.812150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f0e7d-ff6b-4197-baf4-b1b6fa310d0e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:30.848623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94427f-b93d-47a2-bf22-e08560fa488a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:30.848623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94427f-b93d-47a2-bf22-e08560fa488a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:30.848623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94427f-b93d-47a2-bf22-e08560fa488a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:30.848623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 139/272] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:30.855706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d5c7c-96cc-43d6-b4a7-a06a972e545d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:30.856579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d5c7c-96cc-43d6-b4a7-a06a972e545d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:30.857068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d5c7c-96cc-43d6-b4a7-a06a972e545d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:30.867122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd9cb5-9945-447c-88bc-1187c2623eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:30.867971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd9cb5-9945-447c-88bc-1187c2623eee None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:30.868254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd9cb5-9945-447c-88bc-1187c2623eee None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:30.868527 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd9cb5-9945-447c-88bc-1187c2623eee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:30.870000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd9cb5-9945-447c-88bc-1187c2623eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:30.871476 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd9cb5-9945-447c-88bc-1187c2623eee None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:30.873112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 134/273] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:30.878646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd9cb5-9945-447c-88bc-1187c2623eee None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:30.883208 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d02d5c7c-96cc-43d6-b4a7-a06a972e545d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:30.884749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d02d5c7c-96cc-43d6-b4a7-a06a972e545d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:30.996876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 140/274] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:31.008405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd9cb5-9945-447c-88bc-1187c2623eee None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:31.020061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d5c7c-96cc-43d6-b4a7-a06a972e545d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:31.051902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:31.058768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d0c18-3a42-4ee8-b0e4-e98731f28ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:31.059468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d0c18-3a42-4ee8-b0e4-e98731f28ff5 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:31.059752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d0c18-3a42-4ee8-b0e4-e98731f28ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:31.060031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d0c18-3a42-4ee8-b0e4-e98731f28ff5 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:31.060749 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d0c18-3a42-4ee8-b0e4-e98731f28ff5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:31.062199 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d0c18-3a42-4ee8-b0e4-e98731f28ff5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:31.085081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 135/275] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:53:30 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:53:31.085081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938189a-8679-4c3d-a77a-827fb55b6c43 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:31.085081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938189a-8679-4c3d-a77a-827fb55b6c43 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:31.085081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938189a-8679-4c3d-a77a-827fb55b6c43 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:31.085081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 136/276] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:31.096890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163f090-d443-4c4f-bb23-14ef9f4ae2d9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:31.097178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163f090-d443-4c4f-bb23-14ef9f4ae2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:31.097579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163f090-d443-4c4f-bb23-14ef9f4ae2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:31.101322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:31.119577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c163f090-d443-4c4f-bb23-14ef9f4ae2d9 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:31.120947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c163f090-d443-4c4f-bb23-14ef9f4ae2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:31.145899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d5c7c-96cc-43d6-b4a7-a06a972e545d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:31.186178 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76780b8-1b08-4f2a-9a4e-6b7893efe232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:31.187408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76780b8-1b08-4f2a-9a4e-6b7893efe232 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:31.188012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76780b8-1b08-4f2a-9a4e-6b7893efe232 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:31.188498 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76780b8-1b08-4f2a-9a4e-6b7893efe232 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:31.213259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 137/277] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:31.221203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76780b8-1b08-4f2a-9a4e-6b7893efe232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:31.230128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163f090-d443-4c4f-bb23-14ef9f4ae2d9 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-d76780b8-1b08-4f2a-9a4e-6b7893efe232 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:31.233068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:31.234046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:31.234875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:31.234875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:31.234875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:31.234875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:31.234875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:31.234875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:31.234875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:31.234875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:31.234875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:31.234875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:31.234875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:31.234875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:31.237493 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 141/278] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:53:31.246738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76780b8-1b08-4f2a-9a4e-6b7893efe232 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:31.354695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163f090-d443-4c4f-bb23-14ef9f4ae2d9 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:31.375277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:31.382343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76780b8-1b08-4f2a-9a4e-6b7893efe232 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:31.410368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437ccba-08a9-4cc3-8ad3-e0a5cb8867e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:31.411547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437ccba-08a9-4cc3-8ad3-e0a5cb8867e9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:31.412043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437ccba-08a9-4cc3-8ad3-e0a5cb8867e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:31.412527 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437ccba-08a9-4cc3-8ad3-e0a5cb8867e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:31.415054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:31.431196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437ccba-08a9-4cc3-8ad3-e0a5cb8867e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:31.432308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41018a0c-bab1-4710-a3d9-f3cf5220fac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:31.433273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41018a0c-bab1-4710-a3d9-f3cf5220fac2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:31.433562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41018a0c-bab1-4710-a3d9-f3cf5220fac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:31.433847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41018a0c-bab1-4710-a3d9-f3cf5220fac2 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:31.434625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41018a0c-bab1-4710-a3d9-f3cf5220fac2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:31.434938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41018a0c-bab1-4710-a3d9-f3cf5220fac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:31.436378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41018a0c-bab1-4710-a3d9-f3cf5220fac2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-5437ccba-08a9-4cc3-8ad3-e0a5cb8867e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:31.436808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:31.437623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:31.438406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:31.438406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:31.438406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:31.438406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:31.438406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:31.438406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:31.438406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:31.438406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:31.438406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:31.438406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:31.438406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:31.438406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:31.438999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 138/279] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:53:31.445067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437ccba-08a9-4cc3-8ad3-e0a5cb8867e9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:31.447150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 142/280] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:31.454250 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41018a0c-bab1-4710-a3d9-f3cf5220fac2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:31.579534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437ccba-08a9-4cc3-8ad3-e0a5cb8867e9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:31.597805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41018a0c-bab1-4710-a3d9-f3cf5220fac2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:31.618482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172f251-8b21-4bda-bb9b-10519c88b278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:31.619091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172f251-8b21-4bda-bb9b-10519c88b278 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:31.619407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172f251-8b21-4bda-bb9b-10519c88b278 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:31.619683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172f251-8b21-4bda-bb9b-10519c88b278 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:31.620455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172f251-8b21-4bda-bb9b-10519c88b278 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:31.620769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172f251-8b21-4bda-bb9b-10519c88b278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:31.623297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172f251-8b21-4bda-bb9b-10519c88b278 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:31.638714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 139/281] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:31.640625 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172f251-8b21-4bda-bb9b-10519c88b278 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:31.647637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be07c6-c4f5-4887-99c5-cb193106acc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:31.648525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be07c6-c4f5-4887-99c5-cb193106acc6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:31.648810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be07c6-c4f5-4887-99c5-cb193106acc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:31.649088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be07c6-c4f5-4887-99c5-cb193106acc6 None admin] PATH_INFO: `/v3/users/d6e4f831708b472aa8028c61a5902b18` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:31.668288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be07c6-c4f5-4887-99c5-cb193106acc6 None admin] RBAC: Authorizing `identity:get_user(user_id=d6e4f831708b472aa8028c61a5902b18)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:31.669786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be07c6-c4f5-4887-99c5-cb193106acc6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:31.671838 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 143/282] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:31 2022] GET /identity/v3/users/d6e4f831708b472aa8028c61a5902b18 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:31.677422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be07c6-c4f5-4887-99c5-cb193106acc6 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:31.695120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:31.730133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:31.774326 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172f251-8b21-4bda-bb9b-10519c88b278 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:31.794510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be07c6-c4f5-4887-99c5-cb193106acc6 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:31.815342 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b623b-53fe-4386-8204-521848f4daf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:31.816724 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b623b-53fe-4386-8204-521848f4daf4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:31.817237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b623b-53fe-4386-8204-521848f4daf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:31.817725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b623b-53fe-4386-8204-521848f4daf4 None admin] PATH_INFO: `/v3/users/cdd89fadb95241f68d641ef50f4c9021` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:31.819321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b623b-53fe-4386-8204-521848f4daf4 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd89fadb95241f68d641ef50f4c9021)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:31.821211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b623b-53fe-4386-8204-521848f4daf4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:31.827705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 140/283] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:31 2022] GET /identity/v3/users/cdd89fadb95241f68d641ef50f4c9021 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:31.829683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b7662-fcf4-4fcd-8244-d58934f9592c None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:31.830174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b7662-fcf4-4fcd-8244-d58934f9592c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:31.830643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b7662-fcf4-4fcd-8244-d58934f9592c None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:31.832418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 141/284] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:31.834715 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b7662-fcf4-4fcd-8244-d58934f9592c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:31.838716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9a1a8-4f72-4f3f-a44a-63161a44803d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:31.839405 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9a1a8-4f72-4f3f-a44a-63161a44803d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:31.839847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9a1a8-4f72-4f3f-a44a-63161a44803d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:31.840088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9a1a8-4f72-4f3f-a44a-63161a44803d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:31.841312 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9a1a8-4f72-4f3f-a44a-63161a44803d None admin] RBAC: Adding query filter params (role.id=9a562f9a0cad4304852f3183eef398d2, user.id=d6e4f831708b472aa8028c61a5902b18, scope.system=all) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:31.841696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9a1a8-4f72-4f3f-a44a-63161a44803d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:31.843187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9a1a8-4f72-4f3f-a44a-63161a44803d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:31.849050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 144/285] 10.222.0.75 () {56 vars in 1308 bytes} [Thu Jul 14 10:53:31 2022] GET /identity/v3/role_assignments?user.id=d6e4f831708b472aa8028c61a5902b18&scope.system=all&role.id=9a562f9a0cad4304852f3183eef398d2 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:31.850988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7a10c-5bc6-4941-b5e5-f0911f3b1c59 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:31.859537 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7a10c-5bc6-4941-b5e5-f0911f3b1c59 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:31.859983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7a10c-5bc6-4941-b5e5-f0911f3b1c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:31.899177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-17b7a10c-5bc6-4941-b5e5-f0911f3b1c59 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:31.901701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-17b7a10c-5bc6-4941-b5e5-f0911f3b1c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:32.037524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 145/286] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:32.061044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7a10c-5bc6-4941-b5e5-f0911f3b1c59 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:32.062527 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b7662-fcf4-4fcd-8244-d58934f9592c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:32.071135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:32.114176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:32.151092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392de6d-f6e1-43aa-b825-b48d07b3ead8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:32.151910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392de6d-f6e1-43aa-b825-b48d07b3ead8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:32.152245 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392de6d-f6e1-43aa-b825-b48d07b3ead8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:32.152460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392de6d-f6e1-43aa-b825-b48d07b3ead8 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:32.153716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392de6d-f6e1-43aa-b825-b48d07b3ead8 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:32.155161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392de6d-f6e1-43aa-b825-b48d07b3ead8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:32.165857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 142/287] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:31 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:32.175186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392de6d-f6e1-43aa-b825-b48d07b3ead8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:32.202478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7a10c-5bc6-4941-b5e5-f0911f3b1c59 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:32.243845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862e544-edd0-4bdf-b9c7-12fbe4b96655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:32.245023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862e544-edd0-4bdf-b9c7-12fbe4b96655 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:32.245514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862e544-edd0-4bdf-b9c7-12fbe4b96655 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:32.246169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862e544-edd0-4bdf-b9c7-12fbe4b96655 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:32.259909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862e544-edd0-4bdf-b9c7-12fbe4b96655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-3862e544-edd0-4bdf-b9c7-12fbe4b96655 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:32.268620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:32.269528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:32.270317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:32.270317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:32.270317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:32.270317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:32.270317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:32.270317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:32.270317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:32.270317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:32.270317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:32.270317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:32.270317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:32.270317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:32.274491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 146/288] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:53:32.280069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862e544-edd0-4bdf-b9c7-12fbe4b96655 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:32.392375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392de6d-f6e1-43aa-b825-b48d07b3ead8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:32.430802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:32.438787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862e544-edd0-4bdf-b9c7-12fbe4b96655 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:32.465635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918cb42-19fa-4b83-aba6-abfc8e631732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:32.467006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918cb42-19fa-4b83-aba6-abfc8e631732 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:32.467586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918cb42-19fa-4b83-aba6-abfc8e631732 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:32.468157 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918cb42-19fa-4b83-aba6-abfc8e631732 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6/users/cdd89fadb95241f68d641ef50f4c9021/roles/b628b5b418f143f7a1d72f29749e04fd` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:32.473078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918cb42-19fa-4b83-aba6-abfc8e631732 None admin] RBAC: Authorizing `identity:create_grant(project_id=431ab0f9d63c471fb738c37e4c9cbcb6, user_id=cdd89fadb95241f68d641ef50f4c9021, role_id=b628b5b418f143f7a1d72f29749e04fd)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:32.474964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918cb42-19fa-4b83-aba6-abfc8e631732 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:32.481452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad575175-00a4-4f89-9458-b4fc8c215ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:32.481452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad575175-00a4-4f89-9458-b4fc8c215ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:32.481452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad575175-00a4-4f89-9458-b4fc8c215ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:32.481452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad575175-00a4-4f89-9458-b4fc8c215ce9 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:32.482110 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad575175-00a4-4f89-9458-b4fc8c215ce9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:32.483159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad575175-00a4-4f89-9458-b4fc8c215ce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:32.483851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad575175-00a4-4f89-9458-b4fc8c215ce9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:32.499355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 143/289] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:53:32 2022] PUT /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6/users/cdd89fadb95241f68d641ef50f4c9021/roles/b628b5b418f143f7a1d72f29749e04fd => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:32.506503 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 147/290] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:32.513892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918cb42-19fa-4b83-aba6-abfc8e631732 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:32.590756 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:32.667292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918cb42-19fa-4b83-aba6-abfc8e631732 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:32.706694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabb152-e9fc-4224-a57b-7865a6103545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:32.707828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabb152-e9fc-4224-a57b-7865a6103545 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:32.708253 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabb152-e9fc-4224-a57b-7865a6103545 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:32.708696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabb152-e9fc-4224-a57b-7865a6103545 None admin] PATH_INFO: `/v3/users/2dd1dc9757a543b6abf99914d527824c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:32.710079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabb152-e9fc-4224-a57b-7865a6103545 None admin] RBAC: Authorizing `identity:get_user(user_id=2dd1dc9757a543b6abf99914d527824c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:32.711692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabb152-e9fc-4224-a57b-7865a6103545 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:32.713847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 144/291] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:32 2022] GET /identity/v3/users/2dd1dc9757a543b6abf99914d527824c => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:32.719874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad575175-00a4-4f89-9458-b4fc8c215ce9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:32.885078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad575175-00a4-4f89-9458-b4fc8c215ce9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:32.938971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d46550-c0e2-4b30-be93-2ef01673500f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:32.939673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d46550-c0e2-4b30-be93-2ef01673500f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:32.939966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d46550-c0e2-4b30-be93-2ef01673500f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:32.940247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d46550-c0e2-4b30-be93-2ef01673500f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:32.984524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:33.046730 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d46550-c0e2-4b30-be93-2ef01673500f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:33.048238 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d46550-c0e2-4b30-be93-2ef01673500f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:33.212659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 148/292] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:33.218174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabb152-e9fc-4224-a57b-7865a6103545 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:33.336107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabb152-e9fc-4224-a57b-7865a6103545 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:33.337950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:33.383404 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c6c0a-dc20-4a26-aea4-7edd31d1d2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:33.384635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c6c0a-dc20-4a26-aea4-7edd31d1d2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:33.385115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c6c0a-dc20-4a26-aea4-7edd31d1d2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:33.385628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c6c0a-dc20-4a26-aea4-7edd31d1d2b8 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:33.387052 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c6c0a-dc20-4a26-aea4-7edd31d1d2b8 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:33.388762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c6c0a-dc20-4a26-aea4-7edd31d1d2b8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:33.390680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 145/293] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:33 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:33.397047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d46550-c0e2-4b30-be93-2ef01673500f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:33.506280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:33.566172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d46550-c0e2-4b30-be93-2ef01673500f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:33.605179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bef70-7ccb-49b7-990e-6bee5f865479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:33.605756 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bef70-7ccb-49b7-990e-6bee5f865479 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:33.606042 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bef70-7ccb-49b7-990e-6bee5f865479 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:33.606323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bef70-7ccb-49b7-990e-6bee5f865479 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:33.607156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bef70-7ccb-49b7-990e-6bee5f865479 None admin] RBAC: Adding query filter params (role.id=9a562f9a0cad4304852f3183eef398d2, scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6, user.id=2dd1dc9757a543b6abf99914d527824c) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:33.607494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bef70-7ccb-49b7-990e-6bee5f865479 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:33.608955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bef70-7ccb-49b7-990e-6bee5f865479 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:33.615125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 149/294] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:33 2022] GET /identity/v3/role_assignments?user.id=2dd1dc9757a543b6abf99914d527824c&scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6&role.id=9a562f9a0cad4304852f3183eef398d2 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:33.794286 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894f05f-0e82-420e-aa38-da01bc15db20 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:33.795233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894f05f-0e82-420e-aa38-da01bc15db20 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:33.795732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894f05f-0e82-420e-aa38-da01bc15db20 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:33.797317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 146/295] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:33.802818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2e52-5195-4feb-97d5-3d6ea48c322b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:33.803276 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2e52-5195-4feb-97d5-3d6ea48c322b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:33.803490 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2e52-5195-4feb-97d5-3d6ea48c322b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:33.822564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a57a2e52-5195-4feb-97d5-3d6ea48c322b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:33.823817 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a57a2e52-5195-4feb-97d5-3d6ea48c322b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:33.901574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 150/296] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:33.924432 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894f05f-0e82-420e-aa38-da01bc15db20 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:34.051973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894f05f-0e82-420e-aa38-da01bc15db20 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:34.095652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052713d-0fb3-4975-b3dc-80239e1e36cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:34.096459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052713d-0fb3-4975-b3dc-80239e1e36cb None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.096740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052713d-0fb3-4975-b3dc-80239e1e36cb None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.097013 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052713d-0fb3-4975-b3dc-80239e1e36cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.109662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052713d-0fb3-4975-b3dc-80239e1e36cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-8052713d-0fb3-4975-b3dc-80239e1e36cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:34.117993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:34.118855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:34.119651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 147/297] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:34.127239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a2e52-5195-4feb-97d5-3d6ea48c322b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:34.148680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a50fe-47da-4e62-80f3-dbe58876bb02 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.148983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a50fe-47da-4e62-80f3-dbe58876bb02 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.149255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a50fe-47da-4e62-80f3-dbe58876bb02 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.151431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 148/298] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:53:34.157091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58213d2-93c8-426b-a462-38a52421663a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.157571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58213d2-93c8-426b-a462-38a52421663a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.158003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58213d2-93c8-426b-a462-38a52421663a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.175833 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f58213d2-93c8-426b-a462-38a52421663a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:34.177238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f58213d2-93c8-426b-a462-38a52421663a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:34.204712 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:34.214463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:34.262815 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a2e52-5195-4feb-97d5-3d6ea48c322b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:34.275406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 149/299] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:34.298271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58213d2-93c8-426b-a462-38a52421663a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:34.305375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33294c6f-44da-4a0d-8cbd-68a06b2177ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:34.306116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33294c6f-44da-4a0d-8cbd-68a06b2177ba None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.306397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33294c6f-44da-4a0d-8cbd-68a06b2177ba None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.306671 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33294c6f-44da-4a0d-8cbd-68a06b2177ba None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.307757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33294c6f-44da-4a0d-8cbd-68a06b2177ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:34.308158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33294c6f-44da-4a0d-8cbd-68a06b2177ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:34.309667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33294c6f-44da-4a0d-8cbd-68a06b2177ba None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:34.323611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 151/300] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:34.328920 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33294c6f-44da-4a0d-8cbd-68a06b2177ba None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:34.432232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58213d2-93c8-426b-a462-38a52421663a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:34.444308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33294c6f-44da-4a0d-8cbd-68a06b2177ba None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:34.459843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:34.477785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddcaf3-22c3-4b82-896a-9793a9eb2e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:34.478727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddcaf3-22c3-4b82-896a-9793a9eb2e2d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.479014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddcaf3-22c3-4b82-896a-9793a9eb2e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.479294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddcaf3-22c3-4b82-896a-9793a9eb2e2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.480514 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fddcaf3-22c3-4b82-896a-9793a9eb2e2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:34.481979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fddcaf3-22c3-4b82-896a-9793a9eb2e2d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:34.483669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 150/301] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:34.484942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f55fb-d9cc-4b77-a37c-9aec2d0ca45a None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.485223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f55fb-d9cc-4b77-a37c-9aec2d0ca45a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.485498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f55fb-d9cc-4b77-a37c-9aec2d0ca45a None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.486871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 151/302] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:34.491362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab07db5-aefa-4665-8fd4-5764f1a84a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:34.492003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f55fb-d9cc-4b77-a37c-9aec2d0ca45a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:34.492687 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab07db5-aefa-4665-8fd4-5764f1a84a2e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.493170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab07db5-aefa-4665-8fd4-5764f1a84a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.493625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab07db5-aefa-4665-8fd4-5764f1a84a2e None admin] PATH_INFO: `/v3/users/982d55bd43f943a5a47dc7c48e60cfd7` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.510984 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab07db5-aefa-4665-8fd4-5764f1a84a2e None admin] RBAC: Authorizing `identity:get_user(user_id=982d55bd43f943a5a47dc7c48e60cfd7)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:34.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab07db5-aefa-4665-8fd4-5764f1a84a2e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:34.518905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 152/303] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:34 2022] GET /identity/v3/users/982d55bd43f943a5a47dc7c48e60cfd7 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:34.518905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6800668-eea8-482d-99d2-3544bebe8e09 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.518905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6800668-eea8-482d-99d2-3544bebe8e09 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.518905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6800668-eea8-482d-99d2-3544bebe8e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.526784 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:34.533658 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b6800668-eea8-482d-99d2-3544bebe8e09 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:34.534853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b6800668-eea8-482d-99d2-3544bebe8e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:34.606194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f55fb-d9cc-4b77-a37c-9aec2d0ca45a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:34.609165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 153/304] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:34.610020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6800668-eea8-482d-99d2-3544bebe8e09 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:34.659968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b995d-6f00-4b5f-8262-a7db39f96556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:34.660635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b995d-6f00-4b5f-8262-a7db39f96556 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.661408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b995d-6f00-4b5f-8262-a7db39f96556 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.661628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b995d-6f00-4b5f-8262-a7db39f96556 None admin] PATH_INFO: `/v3/groups` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.662888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b995d-6f00-4b5f-8262-a7db39f96556 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:34.665041 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b995d-6f00-4b5f-8262-a7db39f96556 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:34.679645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 152/305] 10.222.0.75 () {60 vars in 1134 bytes} [Thu Jul 14 10:53:34 2022] POST /identity/v3/groups => generated 229 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:53:34.679645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b995d-6f00-4b5f-8262-a7db39f96556 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:34.748366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6800668-eea8-482d-99d2-3544bebe8e09 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:34.751414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:34.799459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:34.805696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5e3c2-5ba7-4d33-9af3-96142eb7dc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:34.806381 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5e3c2-5ba7-4d33-9af3-96142eb7dc0a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.806673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5e3c2-5ba7-4d33-9af3-96142eb7dc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.807018 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5e3c2-5ba7-4d33-9af3-96142eb7dc0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.808234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5e3c2-5ba7-4d33-9af3-96142eb7dc0a None admin] RBAC: Adding query filter params (role.id=b628b5b418f143f7a1d72f29749e04fd, user.id=982d55bd43f943a5a47dc7c48e60cfd7, scope.system=all) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:34.808526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5e3c2-5ba7-4d33-9af3-96142eb7dc0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:34.810559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5e3c2-5ba7-4d33-9af3-96142eb7dc0a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:34.815184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b995d-6f00-4b5f-8262-a7db39f96556 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:34.817756 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 154/306] 10.222.0.75 () {56 vars in 1308 bytes} [Thu Jul 14 10:53:34 2022] GET /identity/v3/role_assignments?user.id=982d55bd43f943a5a47dc7c48e60cfd7&scope.system=all&role.id=b628b5b418f143f7a1d72f29749e04fd => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:34.859029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf058ed9-ef98-4f7b-b48c-35117a479797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:34.860059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf058ed9-ef98-4f7b-b48c-35117a479797 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.860349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf058ed9-ef98-4f7b-b48c-35117a479797 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.860633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf058ed9-ef98-4f7b-b48c-35117a479797 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.873789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf058ed9-ef98-4f7b-b48c-35117a479797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-cf058ed9-ef98-4f7b-b48c-35117a479797 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:34.881977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:34.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:34.883752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 153/307] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:34.889413 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5e3c2-5ba7-4d33-9af3-96142eb7dc0a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:34.948308 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bd279-92fa-45e1-ae49-61c380d9e38e None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.948631 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bd279-92fa-45e1-ae49-61c380d9e38e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.948877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bd279-92fa-45e1-ae49-61c380d9e38e None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.950581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 154/308] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:34.956411 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1615fc-3f08-47c3-9059-cf7fa8e81067 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:34.956701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1615fc-3f08-47c3-9059-cf7fa8e81067 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:34.956984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1615fc-3f08-47c3-9059-cf7fa8e81067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:34.975814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1f1615fc-3f08-47c3-9059-cf7fa8e81067 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:34.977920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1f1615fc-3f08-47c3-9059-cf7fa8e81067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:35.018395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5e3c2-5ba7-4d33-9af3-96142eb7dc0a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:35.054076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 155/309] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:35.061750 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db5b05-5c91-443c-8c6c-cd11f2951a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:35.062425 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db5b05-5c91-443c-8c6c-cd11f2951a3d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:35.062708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db5b05-5c91-443c-8c6c-cd11f2951a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:35.062981 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db5b05-5c91-443c-8c6c-cd11f2951a3d None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:35.063771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db5b05-5c91-443c-8c6c-cd11f2951a3d None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:35.064072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db5b05-5c91-443c-8c6c-cd11f2951a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:35.065587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db5b05-5c91-443c-8c6c-cd11f2951a3d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:35.075606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 155/310] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:35.080716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1615fc-3f08-47c3-9059-cf7fa8e81067 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:35.083008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db5b05-5c91-443c-8c6c-cd11f2951a3d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:35.123803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:35.183636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:35.215335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db5b05-5c91-443c-8c6c-cd11f2951a3d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:35.215708 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1615fc-3f08-47c3-9059-cf7fa8e81067 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:35.251146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3dcfa1-612a-40fd-93b8-ba15ce5d17d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:35.252101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3dcfa1-612a-40fd-93b8-ba15ce5d17d7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:35.252348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3dcfa1-612a-40fd-93b8-ba15ce5d17d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:35.252627 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3dcfa1-612a-40fd-93b8-ba15ce5d17d7 None admin] PATH_INFO: `/v3/users/d6e4f831708b472aa8028c61a5902b18` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:35.253328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e885da-c41f-4431-8184-abf0ce8eb81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:35.253841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3dcfa1-612a-40fd-93b8-ba15ce5d17d7 None admin] RBAC: Authorizing `identity:get_user(user_id=d6e4f831708b472aa8028c61a5902b18)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:35.254555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e885da-c41f-4431-8184-abf0ce8eb81f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:35.255025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e885da-c41f-4431-8184-abf0ce8eb81f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:35.255294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3dcfa1-612a-40fd-93b8-ba15ce5d17d7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:35.256260 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e885da-c41f-4431-8184-abf0ce8eb81f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:35.262400 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 156/311] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:35 2022] GET /identity/v3/users/d6e4f831708b472aa8028c61a5902b18 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:35.263281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3dcfa1-612a-40fd-93b8-ba15ce5d17d7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:35.268826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e885da-c41f-4431-8184-abf0ce8eb81f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-88e885da-c41f-4431-8184-abf0ce8eb81f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:35.277888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:35.278786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:35.279607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:35.279607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:35.279607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:35.279607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:35.279607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:35.279607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:35.279607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:35.279607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:35.279607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:35.279607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:35.279607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:35.279607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:35.282370 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 156/312] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:53:35.288908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e885da-c41f-4431-8184-abf0ce8eb81f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:35.388757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:35.397852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3dcfa1-612a-40fd-93b8-ba15ce5d17d7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:35.403454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e885da-c41f-4431-8184-abf0ce8eb81f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:35.438418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14469409-c8a0-49ef-a7bd-6806847fcce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:35.439783 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14469409-c8a0-49ef-a7bd-6806847fcce5 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:35.440278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14469409-c8a0-49ef-a7bd-6806847fcce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:35.440583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e24a56-2b68-49f9-a425-cea1525811fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:35.440962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14469409-c8a0-49ef-a7bd-6806847fcce5 None admin] PATH_INFO: `/v3/system/users/d6e4f831708b472aa8028c61a5902b18/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:35.441397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e24a56-2b68-49f9-a425-cea1525811fa None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:35.441745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e24a56-2b68-49f9-a425-cea1525811fa None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:35.442081 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e24a56-2b68-49f9-a425-cea1525811fa None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:35.442901 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e24a56-2b68-49f9-a425-cea1525811fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:35.443940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e24a56-2b68-49f9-a425-cea1525811fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:35.444262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14469409-c8a0-49ef-a7bd-6806847fcce5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d6e4f831708b472aa8028c61a5902b18, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:35.445457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e24a56-2b68-49f9-a425-cea1525811fa None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:35.446022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14469409-c8a0-49ef-a7bd-6806847fcce5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:35.456488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 157/313] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:35.459050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 157/314] 10.222.0.75 () {58 vars in 1334 bytes} [Thu Jul 14 10:53:35 2022] PUT /identity/v3/system/users/d6e4f831708b472aa8028c61a5902b18/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:35.462741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e24a56-2b68-49f9-a425-cea1525811fa None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:35.589572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e24a56-2b68-49f9-a425-cea1525811fa None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:35.632770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101e985-7b05-465c-a180-54897ff85145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:35.633652 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101e985-7b05-465c-a180-54897ff85145 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:35.633933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101e985-7b05-465c-a180-54897ff85145 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:35.634206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101e985-7b05-465c-a180-54897ff85145 None admin] PATH_INFO: `/v3/users/cdd89fadb95241f68d641ef50f4c9021` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:35.635494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101e985-7b05-465c-a180-54897ff85145 None admin] RBAC: Authorizing `identity:get_user(user_id=cdd89fadb95241f68d641ef50f4c9021)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:35.636936 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101e985-7b05-465c-a180-54897ff85145 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:35.638748 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 158/315] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:35 2022] GET /identity/v3/users/cdd89fadb95241f68d641ef50f4c9021 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:35.644064 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14469409-c8a0-49ef-a7bd-6806847fcce5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:35.657237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:35.754507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14469409-c8a0-49ef-a7bd-6806847fcce5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:35.792894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea489dd-4f26-4b58-99ce-ec8052022447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:35.793868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea489dd-4f26-4b58-99ce-ec8052022447 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:35.794356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea489dd-4f26-4b58-99ce-ec8052022447 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:35.794804 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea489dd-4f26-4b58-99ce-ec8052022447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:35.870915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea489dd-4f26-4b58-99ce-ec8052022447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:35.873108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea489dd-4f26-4b58-99ce-ec8052022447 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:35.885573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:35.981769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 158/316] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:35.987692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101e985-7b05-465c-a180-54897ff85145 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:36.009615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:36.106191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101e985-7b05-465c-a180-54897ff85145 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:36.153462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa177599-b4d5-4656-8494-45783fde603c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:36.154875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa177599-b4d5-4656-8494-45783fde603c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:36.155343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa177599-b4d5-4656-8494-45783fde603c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:36.155864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa177599-b4d5-4656-8494-45783fde603c None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:36.158169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa177599-b4d5-4656-8494-45783fde603c None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:36.161076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa177599-b4d5-4656-8494-45783fde603c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:36.165124 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 159/317] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:35 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:36.174741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea489dd-4f26-4b58-99ce-ec8052022447 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:36.287618 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea489dd-4f26-4b58-99ce-ec8052022447 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:36.307990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:36.322472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d715edb-e031-409c-87ec-98535d116064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:36.323133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d715edb-e031-409c-87ec-98535d116064 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:36.323431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d715edb-e031-409c-87ec-98535d116064 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:36.323705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d715edb-e031-409c-87ec-98535d116064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:36.324546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d715edb-e031-409c-87ec-98535d116064 None admin] RBAC: Adding query filter params (role.id=b628b5b418f143f7a1d72f29749e04fd, scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6, user.id=cdd89fadb95241f68d641ef50f4c9021) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:36.324859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d715edb-e031-409c-87ec-98535d116064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:36.326315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d715edb-e031-409c-87ec-98535d116064 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:36.333159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 159/318] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:53:36 2022] GET /identity/v3/role_assignments?user.id=cdd89fadb95241f68d641ef50f4c9021&scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6&role.id=b628b5b418f143f7a1d72f29749e04fd => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:36.897551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4c2b8-8669-4b61-a071-1feeac66ebc1 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:36.897929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4c2b8-8669-4b61-a071-1feeac66ebc1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:36.898107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4c2b8-8669-4b61-a071-1feeac66ebc1 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:36.899548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 160/319] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:36.904658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b726a7f-09d4-42dd-82de-88e5e80a5aee None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:36.905112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b726a7f-09d4-42dd-82de-88e5e80a5aee None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:36.905345 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b726a7f-09d4-42dd-82de-88e5e80a5aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:36.922302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8b726a7f-09d4-42dd-82de-88e5e80a5aee None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:36.922770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93315163-b37c-4087-a46d-22818cfa76c4 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:36.923065 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93315163-b37c-4087-a46d-22818cfa76c4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:36.923350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93315163-b37c-4087-a46d-22818cfa76c4 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:36.923540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8b726a7f-09d4-42dd-82de-88e5e80a5aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:36.924874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 161/320] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:36.930225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46dbaa-db10-4df6-9823-02f18925c034 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:36.930518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46dbaa-db10-4df6-9823-02f18925c034 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:36.930798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46dbaa-db10-4df6-9823-02f18925c034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:36.947928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4f46dbaa-db10-4df6-9823-02f18925c034 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:36.949147 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4f46dbaa-db10-4df6-9823-02f18925c034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:37.001107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 160/321] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:37.020549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 162/322] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:37.024251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b726a7f-09d4-42dd-82de-88e5e80a5aee None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:37.041999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46dbaa-db10-4df6-9823-02f18925c034 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:37.061795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:37.145590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b726a7f-09d4-42dd-82de-88e5e80a5aee None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:37.151830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:37.165766 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46dbaa-db10-4df6-9823-02f18925c034 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:37.195022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bc908-2ab7-40dd-a6dd-f57663a99501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:37.195934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bc908-2ab7-40dd-a6dd-f57663a99501 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:37.196224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bc908-2ab7-40dd-a6dd-f57663a99501 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:37.196507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bc908-2ab7-40dd-a6dd-f57663a99501 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:37.197691 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bc908-2ab7-40dd-a6dd-f57663a99501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:37.199188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bc908-2ab7-40dd-a6dd-f57663a99501 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:37.201452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 161/323] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:37.206861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bc908-2ab7-40dd-a6dd-f57663a99501 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:37.216872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c835085-0c07-4488-bba7-856641896f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:37.218074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c835085-0c07-4488-bba7-856641896f12 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:37.218434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c835085-0c07-4488-bba7-856641896f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:37.218718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c835085-0c07-4488-bba7-856641896f12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:37.230962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c835085-0c07-4488-bba7-856641896f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-9c835085-0c07-4488-bba7-856641896f12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:37.239443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:37.240950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:37.241886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 163/324] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:53:37.246899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c835085-0c07-4488-bba7-856641896f12 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:37.315575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bc908-2ab7-40dd-a6dd-f57663a99501 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:37.327689 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:37.352044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c835085-0c07-4488-bba7-856641896f12 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:37.355723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ccafb-efe8-4883-a4cd-72e0ce429f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:37.356648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ccafb-efe8-4883-a4cd-72e0ce429f1c None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:37.357130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ccafb-efe8-4883-a4cd-72e0ce429f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:37.357617 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ccafb-efe8-4883-a4cd-72e0ce429f1c None admin] PATH_INFO: `/v3/groups` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:37.358516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ccafb-efe8-4883-a4cd-72e0ce429f1c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:37.360342 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ccafb-efe8-4883-a4cd-72e0ce429f1c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:37.380054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 162/325] 10.222.0.75 () {60 vars in 1134 bytes} [Thu Jul 14 10:53:37 2022] POST /identity/v3/groups => generated 226 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:53:37.401684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54d520-7aa8-4718-9638-16f63d1f9be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:37.402554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54d520-7aa8-4718-9638-16f63d1f9be2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:37.402853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54d520-7aa8-4718-9638-16f63d1f9be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:37.403202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54d520-7aa8-4718-9638-16f63d1f9be2 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:37.404104 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54d520-7aa8-4718-9638-16f63d1f9be2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:37.404488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54d520-7aa8-4718-9638-16f63d1f9be2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:37.406138 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54d520-7aa8-4718-9638-16f63d1f9be2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:37.416974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 164/326] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:37.422563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ccafb-efe8-4883-a4cd-72e0ce429f1c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:37.459633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:37.551937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ccafb-efe8-4883-a4cd-72e0ce429f1c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:37.597313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772555a-80bb-477c-a7a4-dabfd1e106b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:37.598213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772555a-80bb-477c-a7a4-dabfd1e106b3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:37.598499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772555a-80bb-477c-a7a4-dabfd1e106b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:37.598781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772555a-80bb-477c-a7a4-dabfd1e106b3 None admin] PATH_INFO: `/v3/users/982d55bd43f943a5a47dc7c48e60cfd7` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:37.600714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772555a-80bb-477c-a7a4-dabfd1e106b3 None admin] RBAC: Authorizing `identity:get_user(user_id=982d55bd43f943a5a47dc7c48e60cfd7)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:37.602186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772555a-80bb-477c-a7a4-dabfd1e106b3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:37.604135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 163/327] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:37 2022] GET /identity/v3/users/982d55bd43f943a5a47dc7c48e60cfd7 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:37.609502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54d520-7aa8-4718-9638-16f63d1f9be2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:37.611946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a69fb0-a96b-4d21-b511-5b32abbdac7c None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:37.612242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a69fb0-a96b-4d21-b511-5b32abbdac7c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:37.612528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a69fb0-a96b-4d21-b511-5b32abbdac7c None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:37.614005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 164/328] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:53:37.619013 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b6f70-77f0-4ffe-96ef-762c8b9bb340 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:37.619305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b6f70-77f0-4ffe-96ef-762c8b9bb340 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:37.619699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b6f70-77f0-4ffe-96ef-762c8b9bb340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:37.637024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f03b6f70-77f0-4ffe-96ef-762c8b9bb340 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:37.638229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f03b6f70-77f0-4ffe-96ef-762c8b9bb340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:37.725353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 165/329] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:37.744175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54d520-7aa8-4718-9638-16f63d1f9be2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:37.753545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b6f70-77f0-4ffe-96ef-762c8b9bb340 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:37.790568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bd2c4-6808-425e-a215-ebee79baa779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:37.791480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:37.793249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bd2c4-6808-425e-a215-ebee79baa779 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:37.793758 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bd2c4-6808-425e-a215-ebee79baa779 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:37.794558 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bd2c4-6808-425e-a215-ebee79baa779 None admin] PATH_INFO: `/v3/system/users/982d55bd43f943a5a47dc7c48e60cfd7/roles/b628b5b418f143f7a1d72f29749e04fd` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:37.796937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bd2c4-6808-425e-a215-ebee79baa779 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=982d55bd43f943a5a47dc7c48e60cfd7, role_id=b628b5b418f143f7a1d72f29749e04fd)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:37.799740 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bd2c4-6808-425e-a215-ebee79baa779 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:37.814210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 165/330] 10.222.0.75 () {58 vars in 1334 bytes} [Thu Jul 14 10:53:37 2022] PUT /identity/v3/system/users/982d55bd43f943a5a47dc7c48e60cfd7/roles/b628b5b418f143f7a1d72f29749e04fd => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:37.895048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b6f70-77f0-4ffe-96ef-762c8b9bb340 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:37.941714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420a2c0-6e05-405b-b1b9-d12d130cba56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:37.943832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420a2c0-6e05-405b-b1b9-d12d130cba56 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:37.944325 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420a2c0-6e05-405b-b1b9-d12d130cba56 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:37.944780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420a2c0-6e05-405b-b1b9-d12d130cba56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:37.957033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420a2c0-6e05-405b-b1b9-d12d130cba56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-2420a2c0-6e05-405b-b1b9-d12d130cba56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:37.966066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:37.967038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:37.967835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:37.967835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:37.967835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:37.967835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:37.967835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:37.967835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:37.967835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:37.967835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:37.967835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:37.967835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:37.967835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:37.967835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:37.970365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 166/331] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:53:37.976458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bd2c4-6808-425e-a215-ebee79baa779 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:37.979759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:38.100935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bd2c4-6808-425e-a215-ebee79baa779 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:38.137957 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20371e56-983e-4934-b80c-d9b1a912d2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:38.138636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20371e56-983e-4934-b80c-d9b1a912d2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:38.138921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20371e56-983e-4934-b80c-d9b1a912d2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:38.139203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20371e56-983e-4934-b80c-d9b1a912d2f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:38.139988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20371e56-983e-4934-b80c-d9b1a912d2f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:38.140299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20371e56-983e-4934-b80c-d9b1a912d2f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:38.141690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20371e56-983e-4934-b80c-d9b1a912d2f3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:38.152002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 166/332] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:38.157660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420a2c0-6e05-405b-b1b9-d12d130cba56 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:38.327970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420a2c0-6e05-405b-b1b9-d12d130cba56 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:38.331740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:38.434804 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee4a15-c478-4b7d-83e1-0a3ac534747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:38.435735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4a15-c478-4b7d-83e1-0a3ac534747d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:38.436024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4a15-c478-4b7d-83e1-0a3ac534747d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:38.436307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4a15-c478-4b7d-83e1-0a3ac534747d None admin] PATH_INFO: `/v3/users/d6e4f831708b472aa8028c61a5902b18` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:38.440124 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee4a15-c478-4b7d-83e1-0a3ac534747d None admin] RBAC: Authorizing `identity:get_user(user_id=d6e4f831708b472aa8028c61a5902b18)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:38.441592 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee4a15-c478-4b7d-83e1-0a3ac534747d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:38.444005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 167/333] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:38 2022] GET /identity/v3/users/d6e4f831708b472aa8028c61a5902b18 => generated 293 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:38.449479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20371e56-983e-4934-b80c-d9b1a912d2f3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:38.535553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:38.589692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20371e56-983e-4934-b80c-d9b1a912d2f3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:38.629738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f40c8e-d823-40e0-84d6-2aec485618f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:38.630400 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f40c8e-d823-40e0-84d6-2aec485618f3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:38.631252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f40c8e-d823-40e0-84d6-2aec485618f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:38.631561 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f40c8e-d823-40e0-84d6-2aec485618f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:38.632400 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f40c8e-d823-40e0-84d6-2aec485618f3 None admin] RBAC: Adding query filter params (role.id=9a562f9a0cad4304852f3183eef398d2, user.id=d6e4f831708b472aa8028c61a5902b18, scope.system=all) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:38.632713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f40c8e-d823-40e0-84d6-2aec485618f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:38.634189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f40c8e-d823-40e0-84d6-2aec485618f3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:38.647015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 167/334] 10.222.0.75 () {56 vars in 1308 bytes} [Thu Jul 14 10:53:38 2022] GET /identity/v3/role_assignments?user.id=d6e4f831708b472aa8028c61a5902b18&scope.system=all&role.id=9a562f9a0cad4304852f3183eef398d2 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:40.062725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1f8eb-da13-4df5-bdbe-b7803b7225a7 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:40.063239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1f8eb-da13-4df5-bdbe-b7803b7225a7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:40.063346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1f8eb-da13-4df5-bdbe-b7803b7225a7 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:40.064966 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 168/335] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:40.070982 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f3706-0223-4308-a1f6-0c89d7b9d665 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:40.071877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f3706-0223-4308-a1f6-0c89d7b9d665 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:40.072367 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f3706-0223-4308-a1f6-0c89d7b9d665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:40.094273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ce7f3706-0223-4308-a1f6-0c89d7b9d665 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:40.095877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ce7f3706-0223-4308-a1f6-0c89d7b9d665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:40.176826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 168/336] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:40.219733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1f8eb-da13-4df5-bdbe-b7803b7225a7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:40.348756 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1f8eb-da13-4df5-bdbe-b7803b7225a7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:40.360739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3312ffc-69f6-497b-a9ea-2217b751718c None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:40.361356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3312ffc-69f6-497b-a9ea-2217b751718c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:40.361839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3312ffc-69f6-497b-a9ea-2217b751718c None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:40.363449 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 169/337] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:40.368681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed5ce0-a080-4091-9565-6feedc2d0297 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:40.369255 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed5ce0-a080-4091-9565-6feedc2d0297 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:40.369736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed5ce0-a080-4091-9565-6feedc2d0297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:40.392047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-54ed5ce0-a080-4091-9565-6feedc2d0297 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:40.393613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-54ed5ce0-a080-4091-9565-6feedc2d0297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:40.410006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147478d-2723-4c10-8cbb-8243c89a2b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:40.411198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147478d-2723-4c10-8cbb-8243c89a2b8f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:40.411528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147478d-2723-4c10-8cbb-8243c89a2b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:40.411846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147478d-2723-4c10-8cbb-8243c89a2b8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:40.441875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147478d-2723-4c10-8cbb-8243c89a2b8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-1147478d-2723-4c10-8cbb-8243c89a2b8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:40.452967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:40.453867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:40.454653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:40.454653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:40.454653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:40.454653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:40.454653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:40.454653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:40.454653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:40.454653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:40.454653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:40.454653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:40.454653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:40.454653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:40.466023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 169/338] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:40.475107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147478d-2723-4c10-8cbb-8243c89a2b8f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:40.498505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:40.539221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:40.541457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 170/339] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:40.577446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed5ce0-a080-4091-9565-6feedc2d0297 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:40.691529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147478d-2723-4c10-8cbb-8243c89a2b8f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:40.742191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed5ce0-a080-4091-9565-6feedc2d0297 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:40.748106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae91eb-310c-47ee-90d7-076ff51b0fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:40.749017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae91eb-310c-47ee-90d7-076ff51b0fba None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:40.749451 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae91eb-310c-47ee-90d7-076ff51b0fba None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:40.749888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae91eb-310c-47ee-90d7-076ff51b0fba None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:40.750858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae91eb-310c-47ee-90d7-076ff51b0fba None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:40.751351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae91eb-310c-47ee-90d7-076ff51b0fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:40.752970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae91eb-310c-47ee-90d7-076ff51b0fba None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:40.762955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 170/340] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:40.780455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae91eb-310c-47ee-90d7-076ff51b0fba None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:40.796414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071f375-a5a9-41ed-9079-83f4c54e1387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:40.797492 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071f375-a5a9-41ed-9079-83f4c54e1387 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:40.797953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071f375-a5a9-41ed-9079-83f4c54e1387 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:40.798379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071f375-a5a9-41ed-9079-83f4c54e1387 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:40.818702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071f375-a5a9-41ed-9079-83f4c54e1387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-1071f375-a5a9-41ed-9079-83f4c54e1387 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:40.834528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:40.835551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:40.836374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:40.836374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:40.836374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:40.836374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:40.836374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:40.836374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:40.836374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:40.836374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:40.836374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:40.836374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:40.836374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:40.836374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:40.839283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 171/341] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:40.844877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071f375-a5a9-41ed-9079-83f4c54e1387 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:40.919411 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:40.980107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:41.001644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071f375-a5a9-41ed-9079-83f4c54e1387 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:41.029773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae91eb-310c-47ee-90d7-076ff51b0fba None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:41.043457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421710b-748e-4382-9d3f-dc4189b4358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:41.044505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421710b-748e-4382-9d3f-dc4189b4358a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:41.045020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421710b-748e-4382-9d3f-dc4189b4358a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:41.045500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421710b-748e-4382-9d3f-dc4189b4358a None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:41.046515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421710b-748e-4382-9d3f-dc4189b4358a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:41.046974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421710b-748e-4382-9d3f-dc4189b4358a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:41.048546 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421710b-748e-4382-9d3f-dc4189b4358a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:41.058524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 172/342] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:41.065156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421710b-748e-4382-9d3f-dc4189b4358a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:41.112010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55fafb-d853-4fae-8ef1-8da370727b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:41.113259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55fafb-d853-4fae-8ef1-8da370727b29 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:41.113693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55fafb-d853-4fae-8ef1-8da370727b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:41.114141 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55fafb-d853-4fae-8ef1-8da370727b29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:41.215887 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55fafb-d853-4fae-8ef1-8da370727b29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:41.217748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55fafb-d853-4fae-8ef1-8da370727b29 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:41.292836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421710b-748e-4382-9d3f-dc4189b4358a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:41.296231 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:41.331289 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:41.354096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e70fbf-b40d-4790-95d3-7205c3d58b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:41.355974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e70fbf-b40d-4790-95d3-7205c3d58b40 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:41.356754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e70fbf-b40d-4790-95d3-7205c3d58b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:41.357520 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e70fbf-b40d-4790-95d3-7205c3d58b40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:41.363424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 171/343] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:53:41.366419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588db18-22b6-4f9d-a6ad-5f1cb8835265 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:41.366718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588db18-22b6-4f9d-a6ad-5f1cb8835265 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:41.367053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588db18-22b6-4f9d-a6ad-5f1cb8835265 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:41.371415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 172/344] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:41.371415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588db18-22b6-4f9d-a6ad-5f1cb8835265 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:41.471907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e70fbf-b40d-4790-95d3-7205c3d58b40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:41.473391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e70fbf-b40d-4790-95d3-7205c3d58b40 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:41.488704 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588db18-22b6-4f9d-a6ad-5f1cb8835265 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:41.528452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40285129-41a5-4c01-ba3c-0394fc5dfa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:41.529387 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40285129-41a5-4c01-ba3c-0394fc5dfa7e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:41.529680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40285129-41a5-4c01-ba3c-0394fc5dfa7e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:41.529966 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40285129-41a5-4c01-ba3c-0394fc5dfa7e None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:41.532330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40285129-41a5-4c01-ba3c-0394fc5dfa7e None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:41.533852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40285129-41a5-4c01-ba3c-0394fc5dfa7e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:41.536069 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 173/345] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:41 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:41.538027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6df01-bd21-483f-97da-90925c6954f6 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:41.538320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6df01-bd21-483f-97da-90925c6954f6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:41.538602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6df01-bd21-483f-97da-90925c6954f6 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:41.544714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 174/346] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:41.545944 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebc938-28bd-4056-8657-1fd8a35280c0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:41.546238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebc938-28bd-4056-8657-1fd8a35280c0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:41.546521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebc938-28bd-4056-8657-1fd8a35280c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:41.575553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8eebc938-28bd-4056-8657-1fd8a35280c0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:41.576781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8eebc938-28bd-4056-8657-1fd8a35280c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:41.614165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 173/347] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:53:41.615628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e70fbf-b40d-4790-95d3-7205c3d58b40 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:41.623547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:41.651754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:41.664000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 175/348] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:41.665521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c83d9-6460-486c-a415-e96a68940a4f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:41.665821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c83d9-6460-486c-a415-e96a68940a4f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:41.667000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c83d9-6460-486c-a415-e96a68940a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:41.691812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-117c83d9-6460-486c-a415-e96a68940a4f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:41.694510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-117c83d9-6460-486c-a415-e96a68940a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:41.767118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e70fbf-b40d-4790-95d3-7205c3d58b40 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:41.788874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 176/349] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:41.789752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c83d9-6460-486c-a415-e96a68940a4f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:41.840690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3375dea-f737-4cef-a113-f4daaca452d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:41.841824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3375dea-f737-4cef-a113-f4daaca452d0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:41.842112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3375dea-f737-4cef-a113-f4daaca452d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:41.842393 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3375dea-f737-4cef-a113-f4daaca452d0 None admin] PATH_INFO: `/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:41.852074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3375dea-f737-4cef-a113-f4daaca452d0 None admin] RBAC: Authorizing `identity:get_group(group_id=a5dc4132b9184c7f9ba5044f4c9e538e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:41.854697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3375dea-f737-4cef-a113-f4daaca452d0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:41.857987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 174/350] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:41 2022] GET /identity/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:41.858894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3375dea-f737-4cef-a113-f4daaca452d0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:41.946765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c83d9-6460-486c-a415-e96a68940a4f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:41.957549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:41.966316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:41.991152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8414b7-dc84-4582-9135-c029f55d27b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:41.992305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8414b7-dc84-4582-9135-c029f55d27b6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:41.992734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8414b7-dc84-4582-9135-c029f55d27b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:41.993185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8414b7-dc84-4582-9135-c029f55d27b6 None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:41.994780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8414b7-dc84-4582-9135-c029f55d27b6 None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:41.995989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8414b7-dc84-4582-9135-c029f55d27b6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:41.997716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 177/351] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:41 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:41.998486 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8414b7-dc84-4582-9135-c029f55d27b6 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:42.020603 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3375dea-f737-4cef-a113-f4daaca452d0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:42.074125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576d859-52f6-41e7-a0da-24a3f360368a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:42.074940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576d859-52f6-41e7-a0da-24a3f360368a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:42.075230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576d859-52f6-41e7-a0da-24a3f360368a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:42.075553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576d859-52f6-41e7-a0da-24a3f360368a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:42.090926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576d859-52f6-41e7-a0da-24a3f360368a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-e576d859-52f6-41e7-a0da-24a3f360368a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:42.102962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:42.104517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 175/352] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:42.106032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576d859-52f6-41e7-a0da-24a3f360368a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:42.135289 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8414b7-dc84-4582-9135-c029f55d27b6 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:42.178825 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5822545-f7d7-4f28-9b72-891672e199fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:42.179706 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5822545-f7d7-4f28-9b72-891672e199fd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:42.180049 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5822545-f7d7-4f28-9b72-891672e199fd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:42.180269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5822545-f7d7-4f28-9b72-891672e199fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:42.199276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5822545-f7d7-4f28-9b72-891672e199fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-d5822545-f7d7-4f28-9b72-891672e199fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:42.211008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:42.212193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 178/353] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:42.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee490f92-e544-4a9f-a628-5bcb543e9b28 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:42.213690 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee490f92-e544-4a9f-a628-5bcb543e9b28 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:42.213690 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee490f92-e544-4a9f-a628-5bcb543e9b28 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:42.214595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 179/354] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:42.218755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee490f92-e544-4a9f-a628-5bcb543e9b28 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:42.257336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576d859-52f6-41e7-a0da-24a3f360368a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:42.279966 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:42.316775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:42.337770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47880fe0-368f-4cde-9066-4154c4d02c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:42.338784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47880fe0-368f-4cde-9066-4154c4d02c06 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:42.339252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47880fe0-368f-4cde-9066-4154c4d02c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:42.339759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47880fe0-368f-4cde-9066-4154c4d02c06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:42.340916 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47880fe0-368f-4cde-9066-4154c4d02c06 None admin] RBAC: Adding query filter params (group.id=a5dc4132b9184c7f9ba5044f4c9e538e, role.id=9a562f9a0cad4304852f3183eef398d2, scope.project.id=cc744332e3b04129956875355de3f205) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:42.341488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47880fe0-368f-4cde-9066-4154c4d02c06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:42.344032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47880fe0-368f-4cde-9066-4154c4d02c06 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:42.358644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 176/355] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Jul 14 10:53:42 2022] GET /identity/v3/role_assignments?group.id=a5dc4132b9184c7f9ba5044f4c9e538e&scope.project.id=cc744332e3b04129956875355de3f205&role.id=9a562f9a0cad4304852f3183eef398d2 => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:42.359899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47880fe0-368f-4cde-9066-4154c4d02c06 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:42.406993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee490f92-e544-4a9f-a628-5bcb543e9b28 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:42.460918 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebc434-1a0a-4c65-8df6-b69998be8c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:42.462850 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebc434-1a0a-4c65-8df6-b69998be8c1c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:42.463492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebc434-1a0a-4c65-8df6-b69998be8c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:42.463999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebc434-1a0a-4c65-8df6-b69998be8c1c None admin] PATH_INFO: `/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:42.466018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ebc434-1a0a-4c65-8df6-b69998be8c1c None admin] RBAC: Authorizing `identity:get_group(group_id=a5dc4132b9184c7f9ba5044f4c9e538e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:42.467850 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ebc434-1a0a-4c65-8df6-b69998be8c1c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:42.469888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 180/356] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:42 2022] GET /identity/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:42.471399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88e248-bbef-461b-b55d-930163ad8937 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:42.471903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88e248-bbef-461b-b55d-930163ad8937 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:42.472458 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88e248-bbef-461b-b55d-930163ad8937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:42.491709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2b88e248-bbef-461b-b55d-930163ad8937 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:42.493340 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2b88e248-bbef-461b-b55d-930163ad8937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:42.557424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47880fe0-368f-4cde-9066-4154c4d02c06 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:42.605805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 181/357] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:42.609822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88e248-bbef-461b-b55d-930163ad8937 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:42.618967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39faa6ce-2f50-4b48-8bd8-2e38c6d8fd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:42.619998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39faa6ce-2f50-4b48-8bd8-2e38c6d8fd43 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:42.620511 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39faa6ce-2f50-4b48-8bd8-2e38c6d8fd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:42.620973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39faa6ce-2f50-4b48-8bd8-2e38c6d8fd43 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:42.621966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39faa6ce-2f50-4b48-8bd8-2e38c6d8fd43 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:42.622479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39faa6ce-2f50-4b48-8bd8-2e38c6d8fd43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:42.628280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39faa6ce-2f50-4b48-8bd8-2e38c6d8fd43 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:42.644675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 177/358] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:42.645528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39faa6ce-2f50-4b48-8bd8-2e38c6d8fd43 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:42.656107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:42.723315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:42.765733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88e248-bbef-461b-b55d-930163ad8937 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:42.790697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39faa6ce-2f50-4b48-8bd8-2e38c6d8fd43 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:42.820114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be5cf7-44fe-4d72-8d33-96485a9442c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:42.820803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be5cf7-44fe-4d72-8d33-96485a9442c5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:42.821087 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be5cf7-44fe-4d72-8d33-96485a9442c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:42.821369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be5cf7-44fe-4d72-8d33-96485a9442c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:42.822132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be5cf7-44fe-4d72-8d33-96485a9442c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:42.822448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be5cf7-44fe-4d72-8d33-96485a9442c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:42.823878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be5cf7-44fe-4d72-8d33-96485a9442c5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:42.835315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 182/359] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:42.835839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da80bf5-a98e-41a3-b293-b4ccf2b865b2 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:42.837196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da80bf5-a98e-41a3-b293-b4ccf2b865b2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:42.837196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da80bf5-a98e-41a3-b293-b4ccf2b865b2 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:42.842220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 183/360] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:42.843134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da80bf5-a98e-41a3-b293-b4ccf2b865b2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:42.846873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a89e9-ead9-4cd5-9f62-5f621005fa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:42.847576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a89e9-ead9-4cd5-9f62-5f621005fa05 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:42.847862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a89e9-ead9-4cd5-9f62-5f621005fa05 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:42.849089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a89e9-ead9-4cd5-9f62-5f621005fa05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:42.849089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a89e9-ead9-4cd5-9f62-5f621005fa05 None admin] RBAC: Adding query filter params (group.id=a5dc4132b9184c7f9ba5044f4c9e538e, role.id=ff3de46e4867421f990563b41a305d4d, scope.project.id=cc744332e3b04129956875355de3f205) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:42.849460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a89e9-ead9-4cd5-9f62-5f621005fa05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:42.850720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a89e9-ead9-4cd5-9f62-5f621005fa05 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:42.858325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 178/361] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Jul 14 10:53:42 2022] GET /identity/v3/role_assignments?group.id=a5dc4132b9184c7f9ba5044f4c9e538e&scope.project.id=cc744332e3b04129956875355de3f205&role.id=ff3de46e4867421f990563b41a305d4d => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:42.861637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a89e9-ead9-4cd5-9f62-5f621005fa05 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:42.998938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:43.011106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da80bf5-a98e-41a3-b293-b4ccf2b865b2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:43.016398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a89e9-ead9-4cd5-9f62-5f621005fa05 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:43.040553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:43.065908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc764d3f-aa16-4784-8022-92b2fa9b1f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:43.066287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1317e-f36a-403a-b9e2-23dd4447873e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:43.067195 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1317e-f36a-403a-b9e2-23dd4447873e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:43.067511 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1317e-f36a-403a-b9e2-23dd4447873e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:43.067797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1317e-f36a-403a-b9e2-23dd4447873e None admin] PATH_INFO: `/v3/users/982d55bd43f943a5a47dc7c48e60cfd7` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:43.067906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc764d3f-aa16-4784-8022-92b2fa9b1f1a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:43.069055 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc764d3f-aa16-4784-8022-92b2fa9b1f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:43.069539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc764d3f-aa16-4784-8022-92b2fa9b1f1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:43.069654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1317e-f36a-403a-b9e2-23dd4447873e None admin] RBAC: Authorizing `identity:get_user(user_id=982d55bd43f943a5a47dc7c48e60cfd7)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:43.071134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1317e-f36a-403a-b9e2-23dd4447873e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:43.075415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 179/362] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:53:42 2022] GET /identity/v3/users/982d55bd43f943a5a47dc7c48e60cfd7 => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:43.075415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1317e-f36a-403a-b9e2-23dd4447873e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:43.091407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc764d3f-aa16-4784-8022-92b2fa9b1f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-bc764d3f-aa16-4784-8022-92b2fa9b1f1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:43.103283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:43.104520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:43.105543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:43.105543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:43.105543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:43.105543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:43.105543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:43.105543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:43.105543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:43.105543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:43.105543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:43.105543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:43.105543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:43.105543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:43.109007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 184/363] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:43.111331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e18d9-3824-427e-acca-c601ca343d66 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:43.112201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e18d9-3824-427e-acca-c601ca343d66 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:43.112642 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e18d9-3824-427e-acca-c601ca343d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:43.137722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c88e18d9-3824-427e-acca-c601ca343d66 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:43.139929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c88e18d9-3824-427e-acca-c601ca343d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:43.235365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1317e-f36a-403a-b9e2-23dd4447873e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:43.239987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 185/364] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:43.240807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e18d9-3824-427e-acca-c601ca343d66 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:43.276611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf131c-33c5-4ebe-9d47-70f43bbd5329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:43.277605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf131c-33c5-4ebe-9d47-70f43bbd5329 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:43.278090 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf131c-33c5-4ebe-9d47-70f43bbd5329 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:43.278562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf131c-33c5-4ebe-9d47-70f43bbd5329 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:43.354949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:43.356737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:43.380406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e18d9-3824-427e-acca-c601ca343d66 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:43.383799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf131c-33c5-4ebe-9d47-70f43bbd5329 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:43.385525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf131c-33c5-4ebe-9d47-70f43bbd5329 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:43.435602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744877e-ea38-4963-be4a-e2baf4c7caab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:43.436595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744877e-ea38-4963-be4a-e2baf4c7caab None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:43.437054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744877e-ea38-4963-be4a-e2baf4c7caab None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:43.437899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744877e-ea38-4963-be4a-e2baf4c7caab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:43.439262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744877e-ea38-4963-be4a-e2baf4c7caab None admin] RBAC: Adding query filter params (role.id=b628b5b418f143f7a1d72f29749e04fd, user.id=982d55bd43f943a5a47dc7c48e60cfd7, scope.system=all) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:43.439785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744877e-ea38-4963-be4a-e2baf4c7caab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:43.441414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744877e-ea38-4963-be4a-e2baf4c7caab None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:43.458226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 186/365] 10.222.0.75 () {56 vars in 1308 bytes} [Thu Jul 14 10:53:43 2022] GET /identity/v3/role_assignments?user.id=982d55bd43f943a5a47dc7c48e60cfd7&scope.system=all&role.id=b628b5b418f143f7a1d72f29749e04fd => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:43.460114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744877e-ea38-4963-be4a-e2baf4c7caab None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:43.596660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744877e-ea38-4963-be4a-e2baf4c7caab None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:43.602348 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 180/366] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:43.603500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf131c-33c5-4ebe-9d47-70f43bbd5329 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:43.643005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7982a-c82d-489e-951d-5e3565b01244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:43.644052 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7982a-c82d-489e-951d-5e3565b01244 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:43.644740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7982a-c82d-489e-951d-5e3565b01244 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:43.645348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7982a-c82d-489e-951d-5e3565b01244 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:43.646337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7982a-c82d-489e-951d-5e3565b01244 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:43.646828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7982a-c82d-489e-951d-5e3565b01244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:43.648662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7982a-c82d-489e-951d-5e3565b01244 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:43.659270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 187/367] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:43.661473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7982a-c82d-489e-951d-5e3565b01244 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:43.673559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:43.783839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:43.805653 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf131c-33c5-4ebe-9d47-70f43bbd5329 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:43.806705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7982a-c82d-489e-951d-5e3565b01244 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:43.851933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b735a0-0700-4d8b-be7b-51bffdb79027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:43.852875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b735a0-0700-4d8b-be7b-51bffdb79027 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:43.853226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b735a0-0700-4d8b-be7b-51bffdb79027 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:43.853618 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b735a0-0700-4d8b-be7b-51bffdb79027 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:43.855371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b314330-220c-4a32-89c7-f02ea3c9e3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:43.856984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b314330-220c-4a32-89c7-f02ea3c9e3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:43.857472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b314330-220c-4a32-89c7-f02ea3c9e3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:43.857938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b314330-220c-4a32-89c7-f02ea3c9e3c1 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:43.859612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b314330-220c-4a32-89c7-f02ea3c9e3c1 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:43.862987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b314330-220c-4a32-89c7-f02ea3c9e3c1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:43.866067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 188/368] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:43 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:43.867110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b314330-220c-4a32-89c7-f02ea3c9e3c1 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:43.869336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b735a0-0700-4d8b-be7b-51bffdb79027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-24b735a0-0700-4d8b-be7b-51bffdb79027 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:43.877488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:43.878512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:43.879293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:43.879293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:43.879293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:43.879293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:43.879293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:43.879293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:43.879293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:43.879293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:43.879293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:43.879293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:43.879293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:43.879293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:43.883578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 181/369] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:53:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:43.884577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b735a0-0700-4d8b-be7b-51bffdb79027 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:43.997090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:44.027291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b735a0-0700-4d8b-be7b-51bffdb79027 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:44.033538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b314330-220c-4a32-89c7-f02ea3c9e3c1 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:44.069209 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66796e-4a22-4441-96cd-1a1e5db203e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:44.070020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66796e-4a22-4441-96cd-1a1e5db203e4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:44.070301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66796e-4a22-4441-96cd-1a1e5db203e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:44.070576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66796e-4a22-4441-96cd-1a1e5db203e4 None admin] PATH_INFO: `/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:44.071825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66796e-4a22-4441-96cd-1a1e5db203e4 None admin] RBAC: Authorizing `identity:get_group(group_id=a5dc4132b9184c7f9ba5044f4c9e538e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:44.073252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66796e-4a22-4441-96cd-1a1e5db203e4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:44.074870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 182/370] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:43 2022] GET /identity/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:44.075763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66796e-4a22-4441-96cd-1a1e5db203e4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:44.082557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d514089-d7eb-4798-98e9-b2544cd6ff7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:44.083632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d514089-d7eb-4798-98e9-b2544cd6ff7f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:44.084137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d514089-d7eb-4798-98e9-b2544cd6ff7f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:44.084632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d514089-d7eb-4798-98e9-b2544cd6ff7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:44.094941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:44.201494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d514089-d7eb-4798-98e9-b2544cd6ff7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:44.203587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d514089-d7eb-4798-98e9-b2544cd6ff7f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:44.223633 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66796e-4a22-4441-96cd-1a1e5db203e4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:44.266562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a06fd2-0b02-4256-b20e-4e39861dc092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:44.267160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a06fd2-0b02-4256-b20e-4e39861dc092 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:44.267491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a06fd2-0b02-4256-b20e-4e39861dc092 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:44.267776 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a06fd2-0b02-4256-b20e-4e39861dc092 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:44.268559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a06fd2-0b02-4256-b20e-4e39861dc092 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:44.268838 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a06fd2-0b02-4256-b20e-4e39861dc092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:44.270254 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a06fd2-0b02-4256-b20e-4e39861dc092 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:44.286005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 183/371] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:44.287851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a06fd2-0b02-4256-b20e-4e39861dc092 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:44.308981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:44.348365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 189/372] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:44.350711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d514089-d7eb-4798-98e9-b2544cd6ff7f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:44.423783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:44.431167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a06fd2-0b02-4256-b20e-4e39861dc092 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:44.478826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ec1fd-0fab-41a1-88bf-f9d77d5e3359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:44.479536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ec1fd-0fab-41a1-88bf-f9d77d5e3359 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:44.479825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ec1fd-0fab-41a1-88bf-f9d77d5e3359 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:44.480104 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ec1fd-0fab-41a1-88bf-f9d77d5e3359 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:44.480964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ec1fd-0fab-41a1-88bf-f9d77d5e3359 None admin] RBAC: Adding query filter params (group.id=a5dc4132b9184c7f9ba5044f4c9e538e, role.id=9a562f9a0cad4304852f3183eef398d2, scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:44.481287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ec1fd-0fab-41a1-88bf-f9d77d5e3359 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:44.482757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ec1fd-0fab-41a1-88bf-f9d77d5e3359 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:44.491697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 184/373] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Jul 14 10:53:44 2022] GET /identity/v3/role_assignments?group.id=a5dc4132b9184c7f9ba5044f4c9e538e&scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6&role.id=9a562f9a0cad4304852f3183eef398d2 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:44.492554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ec1fd-0fab-41a1-88bf-f9d77d5e3359 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:44.498628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d514089-d7eb-4798-98e9-b2544cd6ff7f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:44.543809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534968f2-c76c-45e5-be91-26ea74fd3632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:44.544406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534968f2-c76c-45e5-be91-26ea74fd3632 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:44.544689 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534968f2-c76c-45e5-be91-26ea74fd3632 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:44.544969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534968f2-c76c-45e5-be91-26ea74fd3632 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:44.617806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:44.663274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534968f2-c76c-45e5-be91-26ea74fd3632 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:44.666626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534968f2-c76c-45e5-be91-26ea74fd3632 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:44.774846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ec1fd-0fab-41a1-88bf-f9d77d5e3359 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:44.818733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 190/374] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:44.819904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388ee62-f5a5-4012-9caf-2188b02c0c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:44.820778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388ee62-f5a5-4012-9caf-2188b02c0c81 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:44.821066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388ee62-f5a5-4012-9caf-2188b02c0c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:44.821345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388ee62-f5a5-4012-9caf-2188b02c0c81 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:44.822596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388ee62-f5a5-4012-9caf-2188b02c0c81 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:44.824248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388ee62-f5a5-4012-9caf-2188b02c0c81 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:44.824391 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534968f2-c76c-45e5-be91-26ea74fd3632 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:44.826323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 185/375] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:44 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:44.831864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388ee62-f5a5-4012-9caf-2188b02c0c81 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:44.846635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:44.956346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388ee62-f5a5-4012-9caf-2188b02c0c81 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:45.006094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb813a01-5fcd-4f72-9048-5abbab75237b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:45.007350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813a01-5fcd-4f72-9048-5abbab75237b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.007868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813a01-5fcd-4f72-9048-5abbab75237b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.008277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:45.008574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813a01-5fcd-4f72-9048-5abbab75237b None admin] PATH_INFO: `/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.010603 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb813a01-5fcd-4f72-9048-5abbab75237b None admin] RBAC: Authorizing `identity:get_group(group_id=a5dc4132b9184c7f9ba5044f4c9e538e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:45.012604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb813a01-5fcd-4f72-9048-5abbab75237b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:45.015146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 186/376] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:44 2022] GET /identity/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:45.016851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f4ed5-1f6f-4abc-b7c6-c56aaa5f4bc4 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.017352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f4ed5-1f6f-4abc-b7c6-c56aaa5f4bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.017820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f4ed5-1f6f-4abc-b7c6-c56aaa5f4bc4 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.019457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 187/377] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:45.020587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f4ed5-1f6f-4abc-b7c6-c56aaa5f4bc4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:45.045612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534968f2-c76c-45e5-be91-26ea74fd3632 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:45.101848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932663f-b005-4509-a965-97e6c9390c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:45.103063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932663f-b005-4509-a965-97e6c9390c41 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.103613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932663f-b005-4509-a965-97e6c9390c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.104086 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932663f-b005-4509-a965-97e6c9390c41 None admin] PATH_INFO: `/v3/projects/75fd150c50ce43f4ac09652ce03bfda1` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.105607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932663f-b005-4509-a965-97e6c9390c41 None admin] RBAC: Authorizing `identity:get_project(project_id=75fd150c50ce43f4ac09652ce03bfda1)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:45.107375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932663f-b005-4509-a965-97e6c9390c41 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:45.109793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 191/378] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:44 2022] GET /identity/v3/projects/75fd150c50ce43f4ac09652ce03bfda1 => generated 344 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:45.111665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a16a7-0a5d-4052-bc6d-abad986eca20 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.112182 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a16a7-0a5d-4052-bc6d-abad986eca20 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.112765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a16a7-0a5d-4052-bc6d-abad986eca20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.122813 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:45.137193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-506a16a7-0a5d-4052-bc6d-abad986eca20 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:45.138885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-506a16a7-0a5d-4052-bc6d-abad986eca20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:45.153232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f4ed5-1f6f-4abc-b7c6-c56aaa5f4bc4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:45.201979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64607631-d6a2-4927-92ee-88b42ed31ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:45.202165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64607631-d6a2-4927-92ee-88b42ed31ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.202452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64607631-d6a2-4927-92ee-88b42ed31ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.202732 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64607631-d6a2-4927-92ee-88b42ed31ef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.203608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64607631-d6a2-4927-92ee-88b42ed31ef0 None admin] RBAC: Adding query filter params (group.id=a5dc4132b9184c7f9ba5044f4c9e538e, role.id=ff3de46e4867421f990563b41a305d4d, scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:45.203971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64607631-d6a2-4927-92ee-88b42ed31ef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:45.206037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64607631-d6a2-4927-92ee-88b42ed31ef0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:45.214428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 188/379] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Jul 14 10:53:45 2022] GET /identity/v3/role_assignments?group.id=a5dc4132b9184c7f9ba5044f4c9e538e&scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6&role.id=ff3de46e4867421f990563b41a305d4d => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:45.215872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64607631-d6a2-4927-92ee-88b42ed31ef0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:45.238844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 192/380] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:45.270340 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a16a7-0a5d-4052-bc6d-abad986eca20 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:45.358147 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64607631-d6a2-4927-92ee-88b42ed31ef0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:45.370205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:45.404416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2259a-52a1-4a87-9aa5-67c3586d27bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:45.405734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2259a-52a1-4a87-9aa5-67c3586d27bc None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.406320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2259a-52a1-4a87-9aa5-67c3586d27bc None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.406812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2259a-52a1-4a87-9aa5-67c3586d27bc None admin] PATH_INFO: `/v3/groups/f31c181aa5874f93b2b504b2d452e819` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.414570 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a16a7-0a5d-4052-bc6d-abad986eca20 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:45.415066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2259a-52a1-4a87-9aa5-67c3586d27bc None admin] RBAC: Authorizing `identity:get_group(group_id=f31c181aa5874f93b2b504b2d452e819)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:45.418168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2259a-52a1-4a87-9aa5-67c3586d27bc None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:45.420403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 189/381] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:45 2022] GET /identity/v3/groups/f31c181aa5874f93b2b504b2d452e819 => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:45.422237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d73c5-e887-47a3-ab4b-167757652f22 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.422734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d73c5-e887-47a3-ab4b-167757652f22 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.423221 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d73c5-e887-47a3-ab4b-167757652f22 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.425568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 190/382] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:45.428246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d73c5-e887-47a3-ab4b-167757652f22 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:45.434649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:45.465599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6ba0a-87d5-4d03-9d6e-1db3538922ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:45.466824 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6ba0a-87d5-4d03-9d6e-1db3538922ef None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.467317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6ba0a-87d5-4d03-9d6e-1db3538922ef None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.467836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6ba0a-87d5-4d03-9d6e-1db3538922ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.481691 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6ba0a-87d5-4d03-9d6e-1db3538922ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-3ce6ba0a-87d5-4d03-9d6e-1db3538922ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:45.491095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:45.492072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:45.492843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:45.492843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:45.492843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:45.492843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:45.492843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:45.492843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:45.492843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:45.492843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:45.492843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:45.492843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:45.492843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:45.492843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:45.497461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 193/383] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:45.497461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8065dfd-5931-4878-bda4-1aa3e05c0f8f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.497461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8065dfd-5931-4878-bda4-1aa3e05c0f8f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.497461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8065dfd-5931-4878-bda4-1aa3e05c0f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.517014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e8065dfd-5931-4878-bda4-1aa3e05c0f8f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:45.518578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e8065dfd-5931-4878-bda4-1aa3e05c0f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:45.576948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d73c5-e887-47a3-ab4b-167757652f22 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:45.594996 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 194/384] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:45.596353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8065dfd-5931-4878-bda4-1aa3e05c0f8f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:45.639108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8257c5-abc7-48e2-9f64-00941bf9126f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:45.639990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8257c5-abc7-48e2-9f64-00941bf9126f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.640099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8257c5-abc7-48e2-9f64-00941bf9126f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.640353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8257c5-abc7-48e2-9f64-00941bf9126f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.641203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8257c5-abc7-48e2-9f64-00941bf9126f None admin] RBAC: Adding query filter params (group.id=f31c181aa5874f93b2b504b2d452e819, role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=75fd150c50ce43f4ac09652ce03bfda1) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:45.641521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8257c5-abc7-48e2-9f64-00941bf9126f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:45.643241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8257c5-abc7-48e2-9f64-00941bf9126f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:45.651515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 191/385] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Jul 14 10:53:45 2022] GET /identity/v3/role_assignments?group.id=f31c181aa5874f93b2b504b2d452e819&scope.project.id=75fd150c50ce43f4ac09652ce03bfda1&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:45.652529 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8257c5-abc7-48e2-9f64-00941bf9126f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:45.668231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:45.717452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8065dfd-5931-4878-bda4-1aa3e05c0f8f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:45.757552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14036e-1086-4e63-ad8f-999819cad3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:45.758595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14036e-1086-4e63-ad8f-999819cad3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.759111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14036e-1086-4e63-ad8f-999819cad3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.760907 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14036e-1086-4e63-ad8f-999819cad3e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.762051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14036e-1086-4e63-ad8f-999819cad3e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:45.762702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14036e-1086-4e63-ad8f-999819cad3e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:45.764485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14036e-1086-4e63-ad8f-999819cad3e2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:45.775603 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 195/386] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:45.781677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14036e-1086-4e63-ad8f-999819cad3e2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:45.840191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:45.911351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8257c5-abc7-48e2-9f64-00941bf9126f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:45.920906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14036e-1086-4e63-ad8f-999819cad3e2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:45.959805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:45.966288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbc0cc-9360-4c42-94f6-773a36d9fa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:45.967132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbc0cc-9360-4c42-94f6-773a36d9fa2f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.967456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbc0cc-9360-4c42-94f6-773a36d9fa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.967741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbc0cc-9360-4c42-94f6-773a36d9fa2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.975006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860f88-16af-44fb-8e09-1940325716a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:45.979115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860f88-16af-44fb-8e09-1940325716a6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:45.979455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860f88-16af-44fb-8e09-1940325716a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:45.979752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860f88-16af-44fb-8e09-1940325716a6 None admin] PATH_INFO: `/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:45.981388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e860f88-16af-44fb-8e09-1940325716a6 None admin] RBAC: Authorizing `identity:get_group(group_id=a5dc4132b9184c7f9ba5044f4c9e538e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:45.982879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e860f88-16af-44fb-8e09-1940325716a6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:45.984778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 196/387] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:45 2022] GET /identity/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:45.991878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860f88-16af-44fb-8e09-1940325716a6 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:45.993436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbc0cc-9360-4c42-94f6-773a36d9fa2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-a7bbc0cc-9360-4c42-94f6-773a36d9fa2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:46.003787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:46.004711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:46.006068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 192/388] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:46.018318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbc0cc-9360-4c42-94f6-773a36d9fa2f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:46.215506 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbc0cc-9360-4c42-94f6-773a36d9fa2f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:46.233439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860f88-16af-44fb-8e09-1940325716a6 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:46.248850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:46.260439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb961ac-aea0-4b88-a54d-d21cc467fde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:46.261115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb961ac-aea0-4b88-a54d-d21cc467fde8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:46.261429 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb961ac-aea0-4b88-a54d-d21cc467fde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:46.261869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb961ac-aea0-4b88-a54d-d21cc467fde8 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:46.263205 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb961ac-aea0-4b88-a54d-d21cc467fde8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:46.263888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb961ac-aea0-4b88-a54d-d21cc467fde8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:46.265582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb961ac-aea0-4b88-a54d-d21cc467fde8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:46.275803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 193/389] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:46.281473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb961ac-aea0-4b88-a54d-d21cc467fde8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:46.336148 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcb28a-1c62-482a-a1e6-4875f7f27439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:46.337778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcb28a-1c62-482a-a1e6-4875f7f27439 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:46.338272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcb28a-1c62-482a-a1e6-4875f7f27439 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:46.338775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcb28a-1c62-482a-a1e6-4875f7f27439 None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:46.340342 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcb28a-1c62-482a-a1e6-4875f7f27439 None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:46.342052 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcb28a-1c62-482a-a1e6-4875f7f27439 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:46.344113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 197/390] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:45 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:46.350660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcb28a-1c62-482a-a1e6-4875f7f27439 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:46.393863 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb961ac-aea0-4b88-a54d-d21cc467fde8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:46.431942 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dedecd-d908-4482-b0ba-091ca7a1746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:46.433103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dedecd-d908-4482-b0ba-091ca7a1746e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:46.433460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dedecd-d908-4482-b0ba-091ca7a1746e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:46.433721 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dedecd-d908-4482-b0ba-091ca7a1746e None admin] PATH_INFO: `/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:46.434978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dedecd-d908-4482-b0ba-091ca7a1746e None admin] RBAC: Authorizing `identity:get_group(group_id=a5dc4132b9184c7f9ba5044f4c9e538e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:46.436534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dedecd-d908-4482-b0ba-091ca7a1746e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:46.438510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 194/391] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:46 2022] GET /identity/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:46.444435 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dedecd-d908-4482-b0ba-091ca7a1746e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:46.507837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:46.520909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:46.554817 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dedecd-d908-4482-b0ba-091ca7a1746e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:46.592068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c52ab-32d7-4c83-8452-a3de130ad4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:46.592940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c52ab-32d7-4c83-8452-a3de130ad4cf None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:46.593218 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c52ab-32d7-4c83-8452-a3de130ad4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:46.593488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c52ab-32d7-4c83-8452-a3de130ad4cf None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:46.594732 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c52ab-32d7-4c83-8452-a3de130ad4cf None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:46.596223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c52ab-32d7-4c83-8452-a3de130ad4cf None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:46.603813 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 195/392] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:46 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:46.604716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c52ab-32d7-4c83-8452-a3de130ad4cf None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:46.645395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcb28a-1c62-482a-a1e6-4875f7f27439 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:46.691323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2630af-0527-4576-be3a-ea12fabc4cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:46.692466 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2630af-0527-4576-be3a-ea12fabc4cce None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:46.692746 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2630af-0527-4576-be3a-ea12fabc4cce None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:46.693574 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2630af-0527-4576-be3a-ea12fabc4cce None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205/groups/a5dc4132b9184c7f9ba5044f4c9e538e/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:46.695775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2630af-0527-4576-be3a-ea12fabc4cce None admin] RBAC: Authorizing `identity:create_grant(project_id=cc744332e3b04129956875355de3f205, group_id=a5dc4132b9184c7f9ba5044f4c9e538e, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:46.697658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2630af-0527-4576-be3a-ea12fabc4cce None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:46.717764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 198/393] 10.222.0.75 () {58 vars in 1442 bytes} [Thu Jul 14 10:53:46 2022] PUT /identity/v3/projects/cc744332e3b04129956875355de3f205/groups/a5dc4132b9184c7f9ba5044f4c9e538e/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:46.758294 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c52ab-32d7-4c83-8452-a3de130ad4cf None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:46.788357 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:46.795940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b3293-28e5-4808-bd96-1b807e32767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:46.797050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b3293-28e5-4808-bd96-1b807e32767c None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:46.797333 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b3293-28e5-4808-bd96-1b807e32767c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:46.797609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b3293-28e5-4808-bd96-1b807e32767c None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205/groups/a5dc4132b9184c7f9ba5044f4c9e538e/roles/ff3de46e4867421f990563b41a305d4d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:46.799941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b3293-28e5-4808-bd96-1b807e32767c None admin] RBAC: Authorizing `identity:create_grant(project_id=cc744332e3b04129956875355de3f205, group_id=a5dc4132b9184c7f9ba5044f4c9e538e, role_id=ff3de46e4867421f990563b41a305d4d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:46.801399 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b3293-28e5-4808-bd96-1b807e32767c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:46.826078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 196/394] 10.222.0.75 () {58 vars in 1442 bytes} [Thu Jul 14 10:53:46 2022] PUT /identity/v3/projects/cc744332e3b04129956875355de3f205/groups/a5dc4132b9184c7f9ba5044f4c9e538e/roles/ff3de46e4867421f990563b41a305d4d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:47.134400 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04d6c8-4091-40f5-bd02-90fb595f1cab None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:47.134690 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04d6c8-4091-40f5-bd02-90fb595f1cab None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:47.134974 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04d6c8-4091-40f5-bd02-90fb595f1cab None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:47.136656 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 199/395] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:47.153514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ee1e7-a75c-431b-8157-cab936008675 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:47.154123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ee1e7-a75c-431b-8157-cab936008675 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:47.154608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ee1e7-a75c-431b-8157-cab936008675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:47.178736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ac3ee1e7-a75c-431b-8157-cab936008675 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:47.180436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ac3ee1e7-a75c-431b-8157-cab936008675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:47.272863 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 197/396] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:47.309332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04d6c8-4091-40f5-bd02-90fb595f1cab None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:47.453733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04d6c8-4091-40f5-bd02-90fb595f1cab None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:47.476383 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:47.491834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bd670-034d-4bf4-9770-5a62a4d5e386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:47.492654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bd670-034d-4bf4-9770-5a62a4d5e386 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:47.492938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bd670-034d-4bf4-9770-5a62a4d5e386 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:47.493218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bd670-034d-4bf4-9770-5a62a4d5e386 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:47.511925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bd670-034d-4bf4-9770-5a62a4d5e386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-859bd670-034d-4bf4-9770-5a62a4d5e386 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:47.520113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:47.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:47.522840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 200/397] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:47.532544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ee1e7-a75c-431b-8157-cab936008675 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:47.672012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ee1e7-a75c-431b-8157-cab936008675 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:47.691444 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:47.714700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a822a93-2169-4dad-9d64-2a4a1fd5b7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:47.715871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a822a93-2169-4dad-9d64-2a4a1fd5b7ef None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:47.716177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a822a93-2169-4dad-9d64-2a4a1fd5b7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:47.716486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a822a93-2169-4dad-9d64-2a4a1fd5b7ef None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:47.717320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a822a93-2169-4dad-9d64-2a4a1fd5b7ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:47.717702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a822a93-2169-4dad-9d64-2a4a1fd5b7ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:47.719512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a822a93-2169-4dad-9d64-2a4a1fd5b7ef None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:47.729261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 198/398] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:47.735059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bd670-034d-4bf4-9770-5a62a4d5e386 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:47.870234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bd670-034d-4bf4-9770-5a62a4d5e386 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:47.913880 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1c6af-e2e0-4252-a086-39a97adbb148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:47.914749 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c6af-e2e0-4252-a086-39a97adbb148 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:47.915037 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c6af-e2e0-4252-a086-39a97adbb148 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:47.915313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c6af-e2e0-4252-a086-39a97adbb148 None admin] PATH_INFO: `/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:47.916536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1c6af-e2e0-4252-a086-39a97adbb148 None admin] RBAC: Authorizing `identity:get_group(group_id=a5dc4132b9184c7f9ba5044f4c9e538e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:47.917980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1c6af-e2e0-4252-a086-39a97adbb148 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:47.919836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 201/399] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:47 2022] GET /identity/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:47.929012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a822a93-2169-4dad-9d64-2a4a1fd5b7ef None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:48.064957 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a822a93-2169-4dad-9d64-2a4a1fd5b7ef None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:48.110679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e3980-d05f-4a97-8f52-3e031374b70c None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:48.111014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e3980-d05f-4a97-8f52-3e031374b70c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:48.111248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e3980-d05f-4a97-8f52-3e031374b70c None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:48.113116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 202/400] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:48.116156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f425b-e3b5-461e-b57b-f1730083ce69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:48.117102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f425b-e3b5-461e-b57b-f1730083ce69 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:48.117390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f425b-e3b5-461e-b57b-f1730083ce69 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:48.117666 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f425b-e3b5-461e-b57b-f1730083ce69 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:48.118528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f7959-913f-45c6-9896-926e62bd0f81 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:48.118822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f7959-913f-45c6-9896-926e62bd0f81 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:48.118931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f425b-e3b5-461e-b57b-f1730083ce69 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:48.119099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f7959-913f-45c6-9896-926e62bd0f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:48.120448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f425b-e3b5-461e-b57b-f1730083ce69 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:48.122319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 199/401] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:47 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:48.127948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f425b-e3b5-461e-b57b-f1730083ce69 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:48.141211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-646f7959-913f-45c6-9896-926e62bd0f81 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:48.142418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-646f7959-913f-45c6-9896-926e62bd0f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:48.227292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 203/402] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:48.227940 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:48.240333 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:48.251925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f7959-913f-45c6-9896-926e62bd0f81 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:48.373258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f7959-913f-45c6-9896-926e62bd0f81 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:48.410642 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b11ee3-1bdd-499d-b4f9-aeed81243db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:48.411497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b11ee3-1bdd-499d-b4f9-aeed81243db6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:48.412347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b11ee3-1bdd-499d-b4f9-aeed81243db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:48.412630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b11ee3-1bdd-499d-b4f9-aeed81243db6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:48.422640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f425b-e3b5-461e-b57b-f1730083ce69 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:48.432113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b11ee3-1bdd-499d-b4f9-aeed81243db6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-00b11ee3-1bdd-499d-b4f9-aeed81243db6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:48.442257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:48.443872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:48.444648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:48.444648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:48.444648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:48.444648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:48.444648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:48.444648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:48.444648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:48.444648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:48.444648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:48.444648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:48.444648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:48.444648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:48.446651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 204/403] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:53:48.452948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248cdf8-ba69-48de-96ce-a6c74b5f8857 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:48.453451 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248cdf8-ba69-48de-96ce-a6c74b5f8857 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:48.454017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248cdf8-ba69-48de-96ce-a6c74b5f8857 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:48.456288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 205/404] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:48.457147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248cdf8-ba69-48de-96ce-a6c74b5f8857 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:48.483933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba7b48-3532-4c14-b6f1-1028b959f813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:48.485060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba7b48-3532-4c14-b6f1-1028b959f813 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:48.485447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba7b48-3532-4c14-b6f1-1028b959f813 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:48.486000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba7b48-3532-4c14-b6f1-1028b959f813 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6/groups/a5dc4132b9184c7f9ba5044f4c9e538e/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:48.489305 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba7b48-3532-4c14-b6f1-1028b959f813 None admin] RBAC: Authorizing `identity:create_grant(project_id=431ab0f9d63c471fb738c37e4c9cbcb6, group_id=a5dc4132b9184c7f9ba5044f4c9e538e, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:48.491629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba7b48-3532-4c14-b6f1-1028b959f813 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:48.513999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 200/405] 10.222.0.75 () {58 vars in 1442 bytes} [Thu Jul 14 10:53:48 2022] PUT /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6/groups/a5dc4132b9184c7f9ba5044f4c9e538e/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:53:48.515791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40317283-e860-405d-b76d-2fa8c23c6467 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:48.516155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40317283-e860-405d-b76d-2fa8c23c6467 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:48.516506 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40317283-e860-405d-b76d-2fa8c23c6467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:48.537386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-40317283-e860-405d-b76d-2fa8c23c6467 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:48.538901 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-40317283-e860-405d-b76d-2fa8c23c6467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:48.543814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:48.699596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 201/406] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:48.708207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248cdf8-ba69-48de-96ce-a6c74b5f8857 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:48.728248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40317283-e860-405d-b76d-2fa8c23c6467 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:48.757045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e2b62-72e0-4619-b710-171ea4adc04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:48.757744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e2b62-72e0-4619-b710-171ea4adc04a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:48.758029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e2b62-72e0-4619-b710-171ea4adc04a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:48.758371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e2b62-72e0-4619-b710-171ea4adc04a None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:48.759154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e2b62-72e0-4619-b710-171ea4adc04a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:48.759585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e2b62-72e0-4619-b710-171ea4adc04a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:48.763599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e2b62-72e0-4619-b710-171ea4adc04a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:48.776161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 206/407] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:48.782439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e2b62-72e0-4619-b710-171ea4adc04a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:48.785707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:48.912294 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40317283-e860-405d-b76d-2fa8c23c6467 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:48.931818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:48.937234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e2b62-72e0-4619-b710-171ea4adc04a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:49.018224 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd79da-8192-4758-b625-74b28e79befb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:49.020059 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd79da-8192-4758-b625-74b28e79befb None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:49.021473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd79da-8192-4758-b625-74b28e79befb None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:49.023067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd79da-8192-4758-b625-74b28e79befb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:49.030849 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d742450-abf9-4eb4-bc98-25526bddf926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:49.035629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d742450-abf9-4eb4-bc98-25526bddf926 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:49.036499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d742450-abf9-4eb4-bc98-25526bddf926 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:49.038516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d742450-abf9-4eb4-bc98-25526bddf926 None admin] PATH_INFO: `/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:49.043283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d742450-abf9-4eb4-bc98-25526bddf926 None admin] RBAC: Authorizing `identity:get_group(group_id=a5dc4132b9184c7f9ba5044f4c9e538e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:49.046448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d742450-abf9-4eb4-bc98-25526bddf926 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:49.050327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 207/408] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:48 2022] GET /identity/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:49.058950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d742450-abf9-4eb4-bc98-25526bddf926 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:49.061091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd79da-8192-4758-b625-74b28e79befb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-b1bd79da-8192-4758-b625-74b28e79befb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:49.071047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:49.072825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:49.074349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 202/409] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:53:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:53:49.081408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd79da-8192-4758-b625-74b28e79befb None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:49.299070 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d742450-abf9-4eb4-bc98-25526bddf926 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:49.302341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:49.310286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd79da-8192-4758-b625-74b28e79befb None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:49.348068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bd61a-fcf2-4834-8cd8-b01340f14fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:49.348767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bd61a-fcf2-4834-8cd8-b01340f14fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:49.349562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bd61a-fcf2-4834-8cd8-b01340f14fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:49.349975 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bd61a-fcf2-4834-8cd8-b01340f14fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:49.350725 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bd61a-fcf2-4834-8cd8-b01340f14fa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:49.351029 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bd61a-fcf2-4834-8cd8-b01340f14fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:49.352671 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bd61a-fcf2-4834-8cd8-b01340f14fa7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:49.358693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e6e37-008a-44be-8d86-0a3c0dd3e7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:49.360190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e6e37-008a-44be-8d86-0a3c0dd3e7cf None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:49.360698 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e6e37-008a-44be-8d86-0a3c0dd3e7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:49.361175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e6e37-008a-44be-8d86-0a3c0dd3e7cf None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:49.362092 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 203/410] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:49.362936 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e6e37-008a-44be-8d86-0a3c0dd3e7cf None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:49.363515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccedabb-0201-44a0-8798-3d2fc882c2ab None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:49.363804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccedabb-0201-44a0-8798-3d2fc882c2ab None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:49.364078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccedabb-0201-44a0-8798-3d2fc882c2ab None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:49.365868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e6e37-008a-44be-8d86-0a3c0dd3e7cf None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:49.368362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 208/412] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:49 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:49.369423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e6e37-008a-44be-8d86-0a3c0dd3e7cf None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:49.370024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 204/412] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:49.371645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b51d1-e337-4047-a14f-b33dd886468a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:49.371932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b51d1-e337-4047-a14f-b33dd886468a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:49.372209 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b51d1-e337-4047-a14f-b33dd886468a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:49.390436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-263b51d1-e337-4047-a14f-b33dd886468a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:49.391781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-263b51d1-e337-4047-a14f-b33dd886468a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:49.416777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:49.475509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 205/413] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:49.476400 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b51d1-e337-4047-a14f-b33dd886468a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:49.501693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e6e37-008a-44be-8d86-0a3c0dd3e7cf None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:49.554292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e48172-17d4-4d05-8ff4-83d405ffca58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:49.555084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e48172-17d4-4d05-8ff4-83d405ffca58 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:49.555361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e48172-17d4-4d05-8ff4-83d405ffca58 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:49.555719 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e48172-17d4-4d05-8ff4-83d405ffca58 None admin] PATH_INFO: `/v3/groups/f31c181aa5874f93b2b504b2d452e819` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:49.557222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e48172-17d4-4d05-8ff4-83d405ffca58 None admin] RBAC: Authorizing `identity:get_group(group_id=f31c181aa5874f93b2b504b2d452e819)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:49.558932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e48172-17d4-4d05-8ff4-83d405ffca58 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:49.561968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 209/414] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:49 2022] GET /identity/v3/groups/f31c181aa5874f93b2b504b2d452e819 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:49.562780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e48172-17d4-4d05-8ff4-83d405ffca58 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:49.593995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:49.607211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b51d1-e337-4047-a14f-b33dd886468a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:49.644999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b2f85-4554-4c97-aec6-e8977ccf69e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:49.646522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b2f85-4554-4c97-aec6-e8977ccf69e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:49.647015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b2f85-4554-4c97-aec6-e8977ccf69e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:49.647596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b2f85-4554-4c97-aec6-e8977ccf69e0 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6/groups/a5dc4132b9184c7f9ba5044f4c9e538e/roles/ff3de46e4867421f990563b41a305d4d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:49.650122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b2f85-4554-4c97-aec6-e8977ccf69e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=431ab0f9d63c471fb738c37e4c9cbcb6, group_id=a5dc4132b9184c7f9ba5044f4c9e538e, role_id=ff3de46e4867421f990563b41a305d4d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:49.651864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b2f85-4554-4c97-aec6-e8977ccf69e0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:49.668873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 206/415] 10.222.0.75 () {58 vars in 1442 bytes} [Thu Jul 14 10:53:49 2022] PUT /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6/groups/a5dc4132b9184c7f9ba5044f4c9e538e/roles/ff3de46e4867421f990563b41a305d4d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:49.670165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b2f85-4554-4c97-aec6-e8977ccf69e0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:49.705252 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e48172-17d4-4d05-8ff4-83d405ffca58 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:49.713761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:49.768713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f378492-de61-4564-9b44-96ae38946581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:49.770863 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f378492-de61-4564-9b44-96ae38946581 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:49.771505 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f378492-de61-4564-9b44-96ae38946581 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:49.772024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f378492-de61-4564-9b44-96ae38946581 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:49.787435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f378492-de61-4564-9b44-96ae38946581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-9f378492-de61-4564-9b44-96ae38946581 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:49.798439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:49.799420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:49.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:49.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:49.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:49.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:49.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:49.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:49.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:49.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:49.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:49.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:49.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:49.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:49.801791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 210/416] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:49.804959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ba5de-e04e-4fc8-8cef-769e607f2f2b None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:49.805189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ba5de-e04e-4fc8-8cef-769e607f2f2b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:49.805471 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ba5de-e04e-4fc8-8cef-769e607f2f2b None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:49.811721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 211/417] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:49 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:49.812914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ba5de-e04e-4fc8-8cef-769e607f2f2b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:49.817169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b2f85-4554-4c97-aec6-e8977ccf69e0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:49.869345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6597b-6988-4d21-947f-914ba352bf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:49.870210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6597b-6988-4d21-947f-914ba352bf8d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:49.870490 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6597b-6988-4d21-947f-914ba352bf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:49.870765 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6597b-6988-4d21-947f-914ba352bf8d None admin] PATH_INFO: `/v3/projects/75fd150c50ce43f4ac09652ce03bfda1` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:49.872313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6597b-6988-4d21-947f-914ba352bf8d None admin] RBAC: Authorizing `identity:get_project(project_id=75fd150c50ce43f4ac09652ce03bfda1)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:49.873775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6597b-6988-4d21-947f-914ba352bf8d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:49.875630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 207/418] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:49 2022] GET /identity/v3/projects/75fd150c50ce43f4ac09652ce03bfda1 => generated 344 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:49.876591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab245bb-ad7c-40c2-9561-95c3193e1f4f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:49.876878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab245bb-ad7c-40c2-9561-95c3193e1f4f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:49.877183 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab245bb-ad7c-40c2-9561-95c3193e1f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:49.897922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-aab245bb-ad7c-40c2-9561-95c3193e1f4f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:49.899351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-aab245bb-ad7c-40c2-9561-95c3193e1f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:49.912356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:49.955629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ba5de-e04e-4fc8-8cef-769e607f2f2b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:49.998321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 208/419] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:49.999848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab245bb-ad7c-40c2-9561-95c3193e1f4f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:50.004154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499e62d-a4ae-460b-84ed-eb3e382ef151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:50.004920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499e62d-a4ae-460b-84ed-eb3e382ef151 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:50.005295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499e62d-a4ae-460b-84ed-eb3e382ef151 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:50.005626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499e62d-a4ae-460b-84ed-eb3e382ef151 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:50.006520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499e62d-a4ae-460b-84ed-eb3e382ef151 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:50.006914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499e62d-a4ae-460b-84ed-eb3e382ef151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:50.008515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499e62d-a4ae-460b-84ed-eb3e382ef151 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:50.018041 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 212/420] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:50.028537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499e62d-a4ae-460b-84ed-eb3e382ef151 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:50.062184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:50.143062 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab245bb-ad7c-40c2-9561-95c3193e1f4f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:50.150654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499e62d-a4ae-460b-84ed-eb3e382ef151 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:50.201456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a6759-75a6-469c-b4fe-5cb815e83909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:50.202718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a6759-75a6-469c-b4fe-5cb815e83909 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:50.202844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b568ab2-5708-4c26-a9d4-10123cd82856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:50.203339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a6759-75a6-469c-b4fe-5cb815e83909 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:50.203851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a6759-75a6-469c-b4fe-5cb815e83909 None admin] PATH_INFO: `/v3/projects/75fd150c50ce43f4ac09652ce03bfda1/groups/f31c181aa5874f93b2b504b2d452e819/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:50.203952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b568ab2-5708-4c26-a9d4-10123cd82856 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:50.204354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b568ab2-5708-4c26-a9d4-10123cd82856 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:50.204803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b568ab2-5708-4c26-a9d4-10123cd82856 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:50.207505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a6759-75a6-469c-b4fe-5cb815e83909 None admin] RBAC: Authorizing `identity:create_grant(project_id=75fd150c50ce43f4ac09652ce03bfda1, group_id=f31c181aa5874f93b2b504b2d452e819, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:50.209324 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a6759-75a6-469c-b4fe-5cb815e83909 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:50.234345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 209/421] 10.222.0.75 () {58 vars in 1442 bytes} [Thu Jul 14 10:53:49 2022] PUT /identity/v3/projects/75fd150c50ce43f4ac09652ce03bfda1/groups/f31c181aa5874f93b2b504b2d452e819/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:53:50.235674 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a6759-75a6-469c-b4fe-5cb815e83909 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:50.246321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:50.361557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a6759-75a6-469c-b4fe-5cb815e83909 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:50.402460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf36638-b727-4c0c-b021-ee98691ea9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:50.404320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf36638-b727-4c0c-b021-ee98691ea9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:50.404853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf36638-b727-4c0c-b021-ee98691ea9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:50.405331 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf36638-b727-4c0c-b021-ee98691ea9e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:50.419788 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf36638-b727-4c0c-b021-ee98691ea9e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:50.425608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b568ab2-5708-4c26-a9d4-10123cd82856 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:50.427094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b568ab2-5708-4c26-a9d4-10123cd82856 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-6cf36638-b727-4c0c-b021-ee98691ea9e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:50.430443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:50.431346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:50.432166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:50.432166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:50.432166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:50.432166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:50.432166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:50.432166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:50.432166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:50.432166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:50.432166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:50.432166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:50.432166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:50.432166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:50.434922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 210/422] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:50.444824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf36638-b727-4c0c-b021-ee98691ea9e8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:50.495352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:50.567498 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:50.606123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 213/423] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:50.610318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf36638-b727-4c0c-b021-ee98691ea9e8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:50.612302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b568ab2-5708-4c26-a9d4-10123cd82856 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:50.650948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f9649-7dfe-4979-8fd3-6a330c31c775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:50.651710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f9649-7dfe-4979-8fd3-6a330c31c775 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:50.651994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f9649-7dfe-4979-8fd3-6a330c31c775 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:50.652308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f9649-7dfe-4979-8fd3-6a330c31c775 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:50.653069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f9649-7dfe-4979-8fd3-6a330c31c775 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:50.653376 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f9649-7dfe-4979-8fd3-6a330c31c775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:50.654777 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f9649-7dfe-4979-8fd3-6a330c31c775 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:50.664259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 211/424] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:50.670267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f9649-7dfe-4979-8fd3-6a330c31c775 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:50.786740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b568ab2-5708-4c26-a9d4-10123cd82856 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:50.814494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:50.825175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56b7d3-2743-48c5-87f1-972e928ce59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:50.826089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56b7d3-2743-48c5-87f1-972e928ce59c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:50.826372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56b7d3-2743-48c5-87f1-972e928ce59c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:50.826648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56b7d3-2743-48c5-87f1-972e928ce59c None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:50.827868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56b7d3-2743-48c5-87f1-972e928ce59c None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:50.829316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56b7d3-2743-48c5-87f1-972e928ce59c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:50.831298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 214/425] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:50 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:50.840684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56b7d3-2743-48c5-87f1-972e928ce59c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:50.848671 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f9649-7dfe-4979-8fd3-6a330c31c775 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:50.875694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:50.884544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b81f06-83fe-4e17-9e5c-ebfe0ce4a562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:50.885281 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b81f06-83fe-4e17-9e5c-ebfe0ce4a562 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:50.885635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b81f06-83fe-4e17-9e5c-ebfe0ce4a562 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:50.885983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b81f06-83fe-4e17-9e5c-ebfe0ce4a562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:50.988928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b81f06-83fe-4e17-9e5c-ebfe0ce4a562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:50.990761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b81f06-83fe-4e17-9e5c-ebfe0ce4a562 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:51.018131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56b7d3-2743-48c5-87f1-972e928ce59c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:51.055985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f179d-e4f2-4def-ae80-6b24679d758f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:51.057418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f179d-e4f2-4def-ae80-6b24679d758f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:51.058632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f179d-e4f2-4def-ae80-6b24679d758f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:51.059091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f179d-e4f2-4def-ae80-6b24679d758f None admin] PATH_INFO: `/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:51.060748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f179d-e4f2-4def-ae80-6b24679d758f None admin] RBAC: Authorizing `identity:get_group(group_id=a5dc4132b9184c7f9ba5044f4c9e538e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:51.063467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f179d-e4f2-4def-ae80-6b24679d758f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:51.071002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 215/426] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:50 2022] GET /identity/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:51.073163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9cb08-0ad4-4c49-9c32-32fe7933d079 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:51.073597 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9cb08-0ad4-4c49-9c32-32fe7933d079 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:51.074441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9cb08-0ad4-4c49-9c32-32fe7933d079 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:51.077597 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 216/427] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:51.078937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9cb08-0ad4-4c49-9c32-32fe7933d079 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:51.111550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 212/428] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:53:51.112862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2861d8-3c62-4fa7-ae44-9acb49b95506 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:51.113151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2861d8-3c62-4fa7-ae44-9acb49b95506 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:51.113430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2861d8-3c62-4fa7-ae44-9acb49b95506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:51.136112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9e2861d8-3c62-4fa7-ae44-9acb49b95506 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:51.136929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9e2861d8-3c62-4fa7-ae44-9acb49b95506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:51.160700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:51.173027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:51.210874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 213/429] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:51.216415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2861d8-3c62-4fa7-ae44-9acb49b95506 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:51.229857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9cb08-0ad4-4c49-9c32-32fe7933d079 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:51.277086 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd22245-53c5-450f-a19e-928dfdd27cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:51.277996 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd22245-53c5-450f-a19e-928dfdd27cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:51.278883 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd22245-53c5-450f-a19e-928dfdd27cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:51.279341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd22245-53c5-450f-a19e-928dfdd27cb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:51.280765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd22245-53c5-450f-a19e-928dfdd27cb8 None admin] RBAC: Adding query filter params (group.id=a5dc4132b9184c7f9ba5044f4c9e538e, role.id=9a562f9a0cad4304852f3183eef398d2, scope.project.id=cc744332e3b04129956875355de3f205) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:51.281539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd22245-53c5-450f-a19e-928dfdd27cb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:51.283440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd22245-53c5-450f-a19e-928dfdd27cb8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:51.292493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 217/430] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Jul 14 10:53:51 2022] GET /identity/v3/role_assignments?group.id=a5dc4132b9184c7f9ba5044f4c9e538e&scope.project.id=cc744332e3b04129956875355de3f205&role.id=9a562f9a0cad4304852f3183eef398d2 => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:51.293608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd22245-53c5-450f-a19e-928dfdd27cb8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:51.397791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2861d8-3c62-4fa7-ae44-9acb49b95506 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:51.451661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd22245-53c5-450f-a19e-928dfdd27cb8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:51.454137 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21dab6a-5719-431e-be87-30818f55746a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:51.455027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dab6a-5719-431e-be87-30818f55746a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:51.455310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dab6a-5719-431e-be87-30818f55746a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:51.455610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dab6a-5719-431e-be87-30818f55746a None admin] PATH_INFO: `/v3/projects/cc744332e3b04129956875355de3f205` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:51.456921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21dab6a-5719-431e-be87-30818f55746a None admin] RBAC: Authorizing `identity:get_project(project_id=cc744332e3b04129956875355de3f205)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:51.458464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21dab6a-5719-431e-be87-30818f55746a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:51.462205 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 214/431] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:51 2022] GET /identity/v3/projects/cc744332e3b04129956875355de3f205 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:51.467487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21dab6a-5719-431e-be87-30818f55746a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:51.499702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:51.502518 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:51.513610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5a425-d104-40db-9fc6-a63b9f048e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:51.514422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5a425-d104-40db-9fc6-a63b9f048e73 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:51.514762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5a425-d104-40db-9fc6-a63b9f048e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:51.515137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5a425-d104-40db-9fc6-a63b9f048e73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:51.527120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5a425-d104-40db-9fc6-a63b9f048e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-86b5a425-d104-40db-9fc6-a63b9f048e73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:51.535499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:51.536395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:51.537954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 218/432] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:51.543090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5a425-d104-40db-9fc6-a63b9f048e73 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:51.586683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21dab6a-5719-431e-be87-30818f55746a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:51.623982 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec32c0-1138-4e4f-8acb-243f3b906baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:51.624866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec32c0-1138-4e4f-8acb-243f3b906baf None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:51.625146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec32c0-1138-4e4f-8acb-243f3b906baf None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:51.625417 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec32c0-1138-4e4f-8acb-243f3b906baf None admin] PATH_INFO: `/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:51.626622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec32c0-1138-4e4f-8acb-243f3b906baf None admin] RBAC: Authorizing `identity:get_group(group_id=a5dc4132b9184c7f9ba5044f4c9e538e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:51.628114 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec32c0-1138-4e4f-8acb-243f3b906baf None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:51.630037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 215/433] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:51 2022] GET /identity/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:51.635833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec32c0-1138-4e4f-8acb-243f3b906baf None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:51.667850 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5a425-d104-40db-9fc6-a63b9f048e73 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:51.712456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36740bf-65b0-4086-a2d0-712ee58a73b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:51.713452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36740bf-65b0-4086-a2d0-712ee58a73b2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:51.713941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36740bf-65b0-4086-a2d0-712ee58a73b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:51.714418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36740bf-65b0-4086-a2d0-712ee58a73b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:51.715349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36740bf-65b0-4086-a2d0-712ee58a73b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:51.715858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36740bf-65b0-4086-a2d0-712ee58a73b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:51.717462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36740bf-65b0-4086-a2d0-712ee58a73b2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:51.727853 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 219/434] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:53:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:51.734237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36740bf-65b0-4086-a2d0-712ee58a73b2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:51.767190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:51.776159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec32c0-1138-4e4f-8acb-243f3b906baf None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:51.796143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:51.814618 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29379fb0-64bd-4274-bc47-e054f0082eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:51.815664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29379fb0-64bd-4274-bc47-e054f0082eaf None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:51.816183 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29379fb0-64bd-4274-bc47-e054f0082eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:51.816663 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29379fb0-64bd-4274-bc47-e054f0082eaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:51.817703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29379fb0-64bd-4274-bc47-e054f0082eaf None admin] RBAC: Adding query filter params (group.id=a5dc4132b9184c7f9ba5044f4c9e538e, role.id=ff3de46e4867421f990563b41a305d4d, scope.project.id=cc744332e3b04129956875355de3f205) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:51.818301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29379fb0-64bd-4274-bc47-e054f0082eaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:51.821193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29379fb0-64bd-4274-bc47-e054f0082eaf None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:51.835531 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 216/435] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Jul 14 10:53:51 2022] GET /identity/v3/role_assignments?group.id=a5dc4132b9184c7f9ba5044f4c9e538e&scope.project.id=cc744332e3b04129956875355de3f205&role.id=ff3de46e4867421f990563b41a305d4d => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:51.859981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36740bf-65b0-4086-a2d0-712ee58a73b2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:51.909599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc5266-6412-490c-91c5-9c5782344726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:51.910281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc5266-6412-490c-91c5-9c5782344726 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:51.910568 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc5266-6412-490c-91c5-9c5782344726 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:51.910851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc5266-6412-490c-91c5-9c5782344726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:51.994471 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc5266-6412-490c-91c5-9c5782344726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:51.995560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc5266-6412-490c-91c5-9c5782344726 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:52.052462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7404c19-cac4-47a0-b1e1-18596e75c711 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:52.052778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7404c19-cac4-47a0-b1e1-18596e75c711 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:52.053028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7404c19-cac4-47a0-b1e1-18596e75c711 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:52.054417 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 217/436] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:53:52.059150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:52.061999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1bf38-83fc-48ef-8570-ea4cc1624085 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:52.062763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1bf38-83fc-48ef-8570-ea4cc1624085 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:52.063294 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1bf38-83fc-48ef-8570-ea4cc1624085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:52.099016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-41d1bf38-83fc-48ef-8570-ea4cc1624085 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:52.102605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-41d1bf38-83fc-48ef-8570-ea4cc1624085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:52.114620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 220/437] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:53:52.118396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc5266-6412-490c-91c5-9c5782344726 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:52.209181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 218/438] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:52.242798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1bf38-83fc-48ef-8570-ea4cc1624085 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:52.271485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc5266-6412-490c-91c5-9c5782344726 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:52.322401 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9020c05-a867-4646-8181-6ec379fd0a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:52.323284 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9020c05-a867-4646-8181-6ec379fd0a31 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:52.323601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9020c05-a867-4646-8181-6ec379fd0a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:52.323879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9020c05-a867-4646-8181-6ec379fd0a31 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:52.325463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9020c05-a867-4646-8181-6ec379fd0a31 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:52.328526 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9020c05-a867-4646-8181-6ec379fd0a31 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:52.330676 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 221/439] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:52 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:52.340272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9020c05-a867-4646-8181-6ec379fd0a31 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:52.359743 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:52.370067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:52.394036 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1bf38-83fc-48ef-8570-ea4cc1624085 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:52.441960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bc583-4070-4967-81a4-cbd178c4453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:52.442876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bc583-4070-4967-81a4-cbd178c4453e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:52.443156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bc583-4070-4967-81a4-cbd178c4453e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:52.443455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bc583-4070-4967-81a4-cbd178c4453e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:52.464564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5bc583-4070-4967-81a4-cbd178c4453e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-2d5bc583-4070-4967-81a4-cbd178c4453e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:52.472825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:52.473729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:52.480241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 219/440] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:52.480882 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9020c05-a867-4646-8181-6ec379fd0a31 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:52.480953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bc583-4070-4967-81a4-cbd178c4453e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:52.513699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a9a73-3142-42f3-9b08-52e119577950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:52.514578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a9a73-3142-42f3-9b08-52e119577950 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:52.514861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a9a73-3142-42f3-9b08-52e119577950 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:52.515142 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a9a73-3142-42f3-9b08-52e119577950 None admin] PATH_INFO: `/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:52.516394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a9a73-3142-42f3-9b08-52e119577950 None admin] RBAC: Authorizing `identity:get_group(group_id=a5dc4132b9184c7f9ba5044f4c9e538e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:52.517850 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a9a73-3142-42f3-9b08-52e119577950 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:52.519904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 222/441] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:52 2022] GET /identity/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:52.525498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a9a73-3142-42f3-9b08-52e119577950 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:52.614430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bc583-4070-4967-81a4-cbd178c4453e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:52.623783 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:52.644641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a9a73-3142-42f3-9b08-52e119577950 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:52.654872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378896f-525d-424b-a16d-de69479eca6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:52.655587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378896f-525d-424b-a16d-de69479eca6e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:52.655884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378896f-525d-424b-a16d-de69479eca6e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:52.656167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378896f-525d-424b-a16d-de69479eca6e None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:52.656938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378896f-525d-424b-a16d-de69479eca6e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:52.657249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378896f-525d-424b-a16d-de69479eca6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:52.658647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378896f-525d-424b-a16d-de69479eca6e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:52.668795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 220/442] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:52.674339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378896f-525d-424b-a16d-de69479eca6e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:52.687122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:52.690018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b119113-792e-4391-9f64-8ae9ec0887bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:52.691062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b119113-792e-4391-9f64-8ae9ec0887bb None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:52.691804 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b119113-792e-4391-9f64-8ae9ec0887bb None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:52.691915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b119113-792e-4391-9f64-8ae9ec0887bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:52.692794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b119113-792e-4391-9f64-8ae9ec0887bb None admin] RBAC: Adding query filter params (group.id=a5dc4132b9184c7f9ba5044f4c9e538e, role.id=9a562f9a0cad4304852f3183eef398d2, scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:52.693131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b119113-792e-4391-9f64-8ae9ec0887bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:52.694549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b119113-792e-4391-9f64-8ae9ec0887bb None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:52.704084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 223/443] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Jul 14 10:53:52 2022] GET /identity/v3/role_assignments?group.id=a5dc4132b9184c7f9ba5044f4c9e538e&scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6&role.id=9a562f9a0cad4304852f3183eef398d2 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:52.705397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1f966-5b30-49db-ba9d-1c36b4c61fde None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:52.705703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1f966-5b30-49db-ba9d-1c36b4c61fde None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:52.705991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1f966-5b30-49db-ba9d-1c36b4c61fde None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:52.707421 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 224/444] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:52.713688 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054933b0-cdee-4ade-b9d0-3900fc8d649d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:52.713995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054933b0-cdee-4ade-b9d0-3900fc8d649d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:52.714259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054933b0-cdee-4ade-b9d0-3900fc8d649d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:52.735052 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-054933b0-cdee-4ade-b9d0-3900fc8d649d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:52.736445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-054933b0-cdee-4ade-b9d0-3900fc8d649d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:52.806297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378896f-525d-424b-a16d-de69479eca6e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:52.827413 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 225/445] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:52.846325 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054933b0-cdee-4ade-b9d0-3900fc8d649d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:52.846864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037a4b7-3907-4c17-8344-120c35f2af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:52.847594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037a4b7-3907-4c17-8344-120c35f2af2a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:52.847885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037a4b7-3907-4c17-8344-120c35f2af2a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:52.848164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037a4b7-3907-4c17-8344-120c35f2af2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:52.948337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037a4b7-3907-4c17-8344-120c35f2af2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:52.951027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037a4b7-3907-4c17-8344-120c35f2af2a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:52.953153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:52.969334 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:52.986663 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054933b0-cdee-4ade-b9d0-3900fc8d649d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:53.034884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5a539-5de0-4bbf-9e1d-4a0b112b753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:53.035933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5a539-5de0-4bbf-9e1d-4a0b112b753a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:53.036220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5a539-5de0-4bbf-9e1d-4a0b112b753a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:53.036534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5a539-5de0-4bbf-9e1d-4a0b112b753a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:53.051353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5a539-5de0-4bbf-9e1d-4a0b112b753a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-f1d5a539-5de0-4bbf-9e1d-4a0b112b753a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:53.059607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:53.060495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:53.061259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 226/446] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:53:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:53:53.068780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5a539-5de0-4bbf-9e1d-4a0b112b753a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:53.095818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 221/447] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:53:53.101584 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037a4b7-3907-4c17-8344-120c35f2af2a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:53.189744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5a539-5de0-4bbf-9e1d-4a0b112b753a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:53.213383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037a4b7-3907-4c17-8344-120c35f2af2a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:53.224977 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:53.230119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6aeff4-9349-4a35-9347-6dd9e9d3c968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:53.230793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6aeff4-9349-4a35-9347-6dd9e9d3c968 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:53.231081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6aeff4-9349-4a35-9347-6dd9e9d3c968 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:53.231356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6aeff4-9349-4a35-9347-6dd9e9d3c968 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:53.232141 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6aeff4-9349-4a35-9347-6dd9e9d3c968 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:53.232450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6aeff4-9349-4a35-9347-6dd9e9d3c968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:53.233844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6aeff4-9349-4a35-9347-6dd9e9d3c968 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:53.239771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:53.243969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 227/448] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:53:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:53.249303 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6aeff4-9349-4a35-9347-6dd9e9d3c968 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:53.260536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae60ad5-bbb7-4c5e-9bd5-2903c530b777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:53.261921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae60ad5-bbb7-4c5e-9bd5-2903c530b777 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:53.261921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae60ad5-bbb7-4c5e-9bd5-2903c530b777 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:53.262105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae60ad5-bbb7-4c5e-9bd5-2903c530b777 None admin] PATH_INFO: `/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:53.263175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae60ad5-bbb7-4c5e-9bd5-2903c530b777 None admin] RBAC: Authorizing `identity:get_project(project_id=431ab0f9d63c471fb738c37e4c9cbcb6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:53.265033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae60ad5-bbb7-4c5e-9bd5-2903c530b777 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:53.266701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 222/449] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:53 2022] GET /identity/v3/projects/431ab0f9d63c471fb738c37e4c9cbcb6 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:53.273457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae60ad5-bbb7-4c5e-9bd5-2903c530b777 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:53.367135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6aeff4-9349-4a35-9347-6dd9e9d3c968 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:53.382841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae60ad5-bbb7-4c5e-9bd5-2903c530b777 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:53.404842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb669e-440f-44a0-aec9-7b4fc0773691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:53.405503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb669e-440f-44a0-aec9-7b4fc0773691 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:53.405781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb669e-440f-44a0-aec9-7b4fc0773691 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:53.406055 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb669e-440f-44a0-aec9-7b4fc0773691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:53.423829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680fea0-45d2-40ea-b524-656396de6b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:53.424696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680fea0-45d2-40ea-b524-656396de6b62 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:53.424980 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680fea0-45d2-40ea-b524-656396de6b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:53.425261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680fea0-45d2-40ea-b524-656396de6b62 None admin] PATH_INFO: `/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:53.426648 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680fea0-45d2-40ea-b524-656396de6b62 None admin] RBAC: Authorizing `identity:get_group(group_id=a5dc4132b9184c7f9ba5044f4c9e538e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:53.428729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680fea0-45d2-40ea-b524-656396de6b62 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:53.430388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 223/450] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:53 2022] GET /identity/v3/groups/a5dc4132b9184c7f9ba5044f4c9e538e => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:53.436521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680fea0-45d2-40ea-b524-656396de6b62 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:53.481772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:53.484745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb669e-440f-44a0-aec9-7b4fc0773691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:53.486542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb669e-440f-44a0-aec9-7b4fc0773691 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:53.490859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:53.552018 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680fea0-45d2-40ea-b524-656396de6b62 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:53.589994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35375f4-6188-47a1-a5f0-d317c7e79148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:53.590738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35375f4-6188-47a1-a5f0-d317c7e79148 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:53.591023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35375f4-6188-47a1-a5f0-d317c7e79148 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:53.591306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35375f4-6188-47a1-a5f0-d317c7e79148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:53.592172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35375f4-6188-47a1-a5f0-d317c7e79148 None admin] RBAC: Adding query filter params (group.id=a5dc4132b9184c7f9ba5044f4c9e538e, role.id=ff3de46e4867421f990563b41a305d4d, scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:53.592486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35375f4-6188-47a1-a5f0-d317c7e79148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:53.593945 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35375f4-6188-47a1-a5f0-d317c7e79148 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:53.595155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 228/451] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:53:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:53:53.615420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 224/452] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Jul 14 10:53:53 2022] GET /identity/v3/role_assignments?group.id=a5dc4132b9184c7f9ba5044f4c9e538e&scope.project.id=431ab0f9d63c471fb738c37e4c9cbcb6&role.id=ff3de46e4867421f990563b41a305d4d => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:53.615691 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb669e-440f-44a0-aec9-7b4fc0773691 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:53.712723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:53.716237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb669e-440f-44a0-aec9-7b4fc0773691 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:53.755225 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3570e91-9af0-4185-b92c-0e1a2455f8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:53.756121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3570e91-9af0-4185-b92c-0e1a2455f8dc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:53.756410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3570e91-9af0-4185-b92c-0e1a2455f8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:53.756685 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3570e91-9af0-4185-b92c-0e1a2455f8dc None admin] PATH_INFO: `/v3/projects/75fd150c50ce43f4ac09652ce03bfda1` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:53.757902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3570e91-9af0-4185-b92c-0e1a2455f8dc None admin] RBAC: Authorizing `identity:get_project(project_id=75fd150c50ce43f4ac09652ce03bfda1)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:53.759365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3570e91-9af0-4185-b92c-0e1a2455f8dc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:53.761098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 229/453] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:53:53 2022] GET /identity/v3/projects/75fd150c50ce43f4ac09652ce03bfda1 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:53:53.766648 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35375f4-6188-47a1-a5f0-d317c7e79148 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:53.886411 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35375f4-6188-47a1-a5f0-d317c7e79148 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:53.901221 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:53.930843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe8d21-6f52-40d7-98e7-4cd19340d692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:53.931734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe8d21-6f52-40d7-98e7-4cd19340d692 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:53.932025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe8d21-6f52-40d7-98e7-4cd19340d692 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:53.932308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe8d21-6f52-40d7-98e7-4cd19340d692 None admin] PATH_INFO: `/v3/groups/f31c181aa5874f93b2b504b2d452e819` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:53.933781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe8d21-6f52-40d7-98e7-4cd19340d692 None admin] RBAC: Authorizing `identity:get_group(group_id=f31c181aa5874f93b2b504b2d452e819)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:53.935251 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe8d21-6f52-40d7-98e7-4cd19340d692 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:53.936937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 225/454] 10.222.0.75 () {56 vars in 1180 bytes} [Thu Jul 14 10:53:53 2022] GET /identity/v3/groups/f31c181aa5874f93b2b504b2d452e819 => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:53.943016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3570e91-9af0-4185-b92c-0e1a2455f8dc None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:54.045962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3570e91-9af0-4185-b92c-0e1a2455f8dc None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:54.079996 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8334a0e-4327-4b88-ac72-7b91e024c7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:54.080649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8334a0e-4327-4b88-ac72-7b91e024c7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:54.080932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8334a0e-4327-4b88-ac72-7b91e024c7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:54.081206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8334a0e-4327-4b88-ac72-7b91e024c7b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:54.082030 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8334a0e-4327-4b88-ac72-7b91e024c7b4 None admin] RBAC: Adding query filter params (group.id=f31c181aa5874f93b2b504b2d452e819, role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=75fd150c50ce43f4ac09652ce03bfda1) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:54.082339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8334a0e-4327-4b88-ac72-7b91e024c7b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:54.084366 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8334a0e-4327-4b88-ac72-7b91e024c7b4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:54.090543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 230/455] 10.222.0.75 () {56 vars in 1376 bytes} [Thu Jul 14 10:53:53 2022] GET /identity/v3/role_assignments?group.id=f31c181aa5874f93b2b504b2d452e819&scope.project.id=75fd150c50ce43f4ac09652ce03bfda1&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:56.763443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0bd29-15d5-43b8-9efe-b837a1356e68 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:56.763964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0bd29-15d5-43b8-9efe-b837a1356e68 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:56.764038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0bd29-15d5-43b8-9efe-b837a1356e68 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:56.765610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 226/456] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:56.771372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29931f-c4cb-457c-892b-eef45723c235 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:56.772341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29931f-c4cb-457c-892b-eef45723c235 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:56.772817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29931f-c4cb-457c-892b-eef45723c235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:56.797142 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3a29931f-c4cb-457c-892b-eef45723c235 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:56.799560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3a29931f-c4cb-457c-892b-eef45723c235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:56.832096 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:56.882506 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 231/457] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:56.906790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0bd29-15d5-43b8-9efe-b837a1356e68 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:56.943234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf9d7d-c526-4d11-9d6c-5930eed8963f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:56.943234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf9d7d-c526-4d11-9d6c-5930eed8963f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:56.943234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf9d7d-c526-4d11-9d6c-5930eed8963f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:56.943234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 232/458] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:56.951235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccc8a6-9c92-41f3-b7bd-ea28b6a5a5b3 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:56.951778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccc8a6-9c92-41f3-b7bd-ea28b6a5a5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:56.952236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccc8a6-9c92-41f3-b7bd-ea28b6a5a5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:56.972420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-cfccc8a6-9c92-41f3-b7bd-ea28b6a5a5b3 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:56.974393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cfccc8a6-9c92-41f3-b7bd-ea28b6a5a5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:57.058850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0bd29-15d5-43b8-9efe-b837a1356e68 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:57.059888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 233/459] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:57.061534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41499011-376a-4cc1-9b19-4254ce4a57b6 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:57.062049 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41499011-376a-4cc1-9b19-4254ce4a57b6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:57.062538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41499011-376a-4cc1-9b19-4254ce4a57b6 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:57.064051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 234/460] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:57.069602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c50c43-dd6f-489a-bbcb-da2e27528ad9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:57.070183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c50c43-dd6f-489a-bbcb-da2e27528ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:57.070680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c50c43-dd6f-489a-bbcb-da2e27528ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:57.090341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-04c50c43-dd6f-489a-bbcb-da2e27528ad9 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:57.091929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-04c50c43-dd6f-489a-bbcb-da2e27528ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:57.121122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59a83a-aff4-43e4-af1f-8fdb75292239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:57.121949 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59a83a-aff4-43e4-af1f-8fdb75292239 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:57.122384 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59a83a-aff4-43e4-af1f-8fdb75292239 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:57.122937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59a83a-aff4-43e4-af1f-8fdb75292239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:57.137123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59a83a-aff4-43e4-af1f-8fdb75292239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:57.138772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59a83a-aff4-43e4-af1f-8fdb75292239 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-3d59a83a-aff4-43e4-af1f-8fdb75292239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:53:57.162554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:57.163499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:57.164328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:53:57.164328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:53:57.164328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:53:57.164328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:57.164328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 227/461] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:57.169842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59a83a-aff4-43e4-af1f-8fdb75292239 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:57.204646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 235/462] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:57.205404 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c50c43-dd6f-489a-bbcb-da2e27528ad9 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:57.217844 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:57.245862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:57.305704 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59a83a-aff4-43e4-af1f-8fdb75292239 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:57.344344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c50c43-dd6f-489a-bbcb-da2e27528ad9 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:57.348709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19023fd1-a2b2-4896-b902-97ad9f09f489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:57.350079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19023fd1-a2b2-4896-b902-97ad9f09f489 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:57.350360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19023fd1-a2b2-4896-b902-97ad9f09f489 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:57.350630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19023fd1-a2b2-4896-b902-97ad9f09f489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:57.364591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19023fd1-a2b2-4896-b902-97ad9f09f489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:57.366670 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19023fd1-a2b2-4896-b902-97ad9f09f489 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-19023fd1-a2b2-4896-b902-97ad9f09f489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:53:57.384372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:57.385322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:57.386267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:53:57.386267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:53:57.386267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:53:57.386267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:57.386267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 228/463] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:57.386877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdd96a-525f-44a4-87d2-d33b035528f8 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:57.387087 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdd96a-525f-44a4-87d2-d33b035528f8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:57.387366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdd96a-525f-44a4-87d2-d33b035528f8 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:57.392629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 229/464] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:57.392629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdd96a-525f-44a4-87d2-d33b035528f8 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:57.393679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2b9cc-79e0-42cd-95ce-d86cfaa1b9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:57.394375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2b9cc-79e0-42cd-95ce-d86cfaa1b9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:57.395462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2b9cc-79e0-42cd-95ce-d86cfaa1b9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:57.395462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2b9cc-79e0-42cd-95ce-d86cfaa1b9f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:57.396409 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2b9cc-79e0-42cd-95ce-d86cfaa1b9f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:57.396697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2b9cc-79e0-42cd-95ce-d86cfaa1b9f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:57.398219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2b9cc-79e0-42cd-95ce-d86cfaa1b9f8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:57.417638 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 236/465] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:57.418575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2b9cc-79e0-42cd-95ce-d86cfaa1b9f8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:57.519482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:57.547328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:57.552554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdd96a-525f-44a4-87d2-d33b035528f8 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:57.569604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2b9cc-79e0-42cd-95ce-d86cfaa1b9f8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:57.600481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4156d6-3f4a-4111-bf0f-8a46063182b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:57.601263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4156d6-3f4a-4111-bf0f-8a46063182b8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:57.601453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4156d6-3f4a-4111-bf0f-8a46063182b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:57.601732 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4156d6-3f4a-4111-bf0f-8a46063182b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:57.610939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5ea40-3860-4a5f-b703-49f01f1ca28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:57.611646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5ea40-3860-4a5f-b703-49f01f1ca28e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:57.611934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5ea40-3860-4a5f-b703-49f01f1ca28e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:57.612213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5ea40-3860-4a5f-b703-49f01f1ca28e None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:57.612941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5ea40-3860-4a5f-b703-49f01f1ca28e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:57.613255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5ea40-3860-4a5f-b703-49f01f1ca28e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:57.613491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4156d6-3f4a-4111-bf0f-8a46063182b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:57.614723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5ea40-3860-4a5f-b703-49f01f1ca28e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:57.615007 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4156d6-3f4a-4111-bf0f-8a46063182b8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:57.630072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 237/466] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:57.632214 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b80ab-0bd6-4590-82f8-90382c1cdaa0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:57.632508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b80ab-0bd6-4590-82f8-90382c1cdaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:57.632792 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b80ab-0bd6-4590-82f8-90382c1cdaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-3d4156d6-3f4a-4111-bf0f-8a46063182b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:53:57.637879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:57.638720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:57.639587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:53:57.639587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:53:57.639587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:53:57.639587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:57.641099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 230/467] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:57.642071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4156d6-3f4a-4111-bf0f-8a46063182b8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:57.652845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3b6b80ab-0bd6-4590-82f8-90382c1cdaa0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:57.654058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3b6b80ab-0bd6-4590-82f8-90382c1cdaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:57.731717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 238/468] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:53:57.732714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b80ab-0bd6-4590-82f8-90382c1cdaa0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:57.780787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4156d6-3f4a-4111-bf0f-8a46063182b8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:57.813964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:57.820668 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287dbf6-dddc-40f1-971d-e690907ee577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:57.821351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287dbf6-dddc-40f1-971d-e690907ee577 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:57.821638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287dbf6-dddc-40f1-971d-e690907ee577 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:57.822242 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287dbf6-dddc-40f1-971d-e690907ee577 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:57.822619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287dbf6-dddc-40f1-971d-e690907ee577 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:57.824142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287dbf6-dddc-40f1-971d-e690907ee577 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:57.831096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:57.855501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 231/469] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:53:57 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:53:57.855501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287dbf6-dddc-40f1-971d-e690907ee577 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:57.867859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b80ab-0bd6-4590-82f8-90382c1cdaa0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:57.916866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df1693-1cc6-4915-b759-24055d4457f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:57.917571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df1693-1cc6-4915-b759-24055d4457f8 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:57.917855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df1693-1cc6-4915-b759-24055d4457f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:57.918136 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df1693-1cc6-4915-b759-24055d4457f8 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:57.919265 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df1693-1cc6-4915-b759-24055d4457f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:57.921473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df1693-1cc6-4915-b759-24055d4457f8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:57.951180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 239/470] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:53:57 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:53:57.956542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df1693-1cc6-4915-b759-24055d4457f8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:58.026082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287dbf6-dddc-40f1-971d-e690907ee577 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:58.078570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d2e8f-1dc2-49cd-bbe9-a686396dc3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:58.079747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d2e8f-1dc2-49cd-bbe9-a686396dc3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:58.080579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d2e8f-1dc2-49cd-bbe9-a686396dc3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:58.081065 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d2e8f-1dc2-49cd-bbe9-a686396dc3a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:58.082783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d2e8f-1dc2-49cd-bbe9-a686396dc3a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:58.083688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d2e8f-1dc2-49cd-bbe9-a686396dc3a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:58.085645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d2e8f-1dc2-49cd-bbe9-a686396dc3a7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:58.103846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 232/471] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:58.114044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d2e8f-1dc2-49cd-bbe9-a686396dc3a7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:58.135918 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df1693-1cc6-4915-b759-24055d4457f8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:58.164540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:58.174690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:58.179231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53641c5-1d13-466a-89c0-282c6015e445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:58.179231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53641c5-1d13-466a-89c0-282c6015e445 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:58.179231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53641c5-1d13-466a-89c0-282c6015e445 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:58.179231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53641c5-1d13-466a-89c0-282c6015e445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:58.201019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53641c5-1d13-466a-89c0-282c6015e445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:58.202515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53641c5-1d13-466a-89c0-282c6015e445 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c53641c5-1d13-466a-89c0-282c6015e445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:53:58.231571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:58.232457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:58.233581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:53:58.233581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:53:58.233581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:53:58.233581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:58.233581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 240/472] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:58.239319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53641c5-1d13-466a-89c0-282c6015e445 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:58.260375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d2e8f-1dc2-49cd-bbe9-a686396dc3a7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:58.315229 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7dc8e-a10b-4a66-8f11-9a9b28024b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:58.315937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7dc8e-a10b-4a66-8f11-9a9b28024b25 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:58.316219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7dc8e-a10b-4a66-8f11-9a9b28024b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:58.316494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7dc8e-a10b-4a66-8f11-9a9b28024b25 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:58.317201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7dc8e-a10b-4a66-8f11-9a9b28024b25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:58.318666 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7dc8e-a10b-4a66-8f11-9a9b28024b25 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:58.344759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 233/473] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:53:58 2022] POST /identity/v3/users => generated 246 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:53:58.346451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6ff09-240c-4585-8010-6d965a846c43 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:58.346870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6ff09-240c-4585-8010-6d965a846c43 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:58.347269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6ff09-240c-4585-8010-6d965a846c43 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:58.354026 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 234/474] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:53:58.356562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6f32f-c8c2-484b-98d6-42ad261278c0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:58.357014 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6f32f-c8c2-484b-98d6-42ad261278c0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:58.357436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6f32f-c8c2-484b-98d6-42ad261278c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:58.394741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6eb6f32f-c8c2-484b-98d6-42ad261278c0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:53:58.396045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6eb6f32f-c8c2-484b-98d6-42ad261278c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:53:58.443833 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53641c5-1d13-466a-89c0-282c6015e445 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:58.515331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ac92b-e7f8-4bc7-aebf-3d85324d475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:58.516048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ac92b-e7f8-4bc7-aebf-3d85324d475b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:58.516334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ac92b-e7f8-4bc7-aebf-3d85324d475b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:58.516614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ac92b-e7f8-4bc7-aebf-3d85324d475b None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:58.517367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ac92b-e7f8-4bc7-aebf-3d85324d475b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:58.517677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ac92b-e7f8-4bc7-aebf-3d85324d475b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:58.519220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ac92b-e7f8-4bc7-aebf-3d85324d475b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:58.543056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 235/475] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:53:58.547643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 241/476] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:53:58.553272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6f32f-c8c2-484b-98d6-42ad261278c0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:58.584758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ac92b-e7f8-4bc7-aebf-3d85324d475b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:58.622359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:58.640896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:58.737696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6f32f-c8c2-484b-98d6-42ad261278c0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:58.746790 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ac92b-e7f8-4bc7-aebf-3d85324d475b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:58.778369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2229301-406d-4929-ae8f-41af8a2c501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:58.779066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2229301-406d-4929-ae8f-41af8a2c501f None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:58.779356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2229301-406d-4929-ae8f-41af8a2c501f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:58.779684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2229301-406d-4929-ae8f-41af8a2c501f None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:58.780403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2229301-406d-4929-ae8f-41af8a2c501f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:58.781871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2229301-406d-4929-ae8f-41af8a2c501f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:58.811970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f9998-6066-4c08-a32d-b7a39e0518e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:58.812653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f9998-6066-4c08-a32d-b7a39e0518e4 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:58.812937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f9998-6066-4c08-a32d-b7a39e0518e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:58.813335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f9998-6066-4c08-a32d-b7a39e0518e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:58.814395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 236/477] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:53:58 2022] POST /identity/v3/users => generated 249 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:53:58.814651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f9998-6066-4c08-a32d-b7a39e0518e4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:58.817061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f9998-6066-4c08-a32d-b7a39e0518e4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:58.831558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.sql.core [None req-df0f9998-6066-4c08-a32d-b7a39e0518e4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 10:53:58.831558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 10:53:58.831558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [parameters: {'id': '72ffe0c63b19448b9528e465519647dd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 10:53:58.831558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80183) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-df0f9998-6066-4c08-a32d-b7a39e0518e4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result.read() Jul 14 10:53:58.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return ref.to_dict() Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:53:58.865057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application yield resource Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise self.value Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application session.commit() Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 10:53:58.865979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.session.flush() Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application compat.raise_( Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 10:53:58.866805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:53:58.868728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result.read() Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:53:58.870747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [parameters: {'id': '72ffe0c63b19448b9528e465519647dd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:58.872770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 10:53:58.875969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:53:58.878027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 242/478] 10.222.0.75 () {60 vars in 1131 bytes} [Thu Jul 14 10:53:58 2022] POST /identity/v3/roles => generated 144 bytes in 282 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:58.878179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2229301-406d-4929-ae8f-41af8a2c501f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:59.036252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:59.047107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2229301-406d-4929-ae8f-41af8a2c501f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:59.084982 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781e671-af81-43ca-9db5-4d935cbf43c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:59.085816 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781e671-af81-43ca-9db5-4d935cbf43c0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:59.086096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781e671-af81-43ca-9db5-4d935cbf43c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:59.086366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781e671-af81-43ca-9db5-4d935cbf43c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:59.098368 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781e671-af81-43ca-9db5-4d935cbf43c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f781e671-af81-43ca-9db5-4d935cbf43c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:53:59.106638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:53:59.107557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:53:59.108350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 237/479] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:53:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:53:59.113727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f9998-6066-4c08-a32d-b7a39e0518e4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:53:59.225081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f9998-6066-4c08-a32d-b7a39e0518e4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:53:59.228040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:59.264695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246917d2-a218-42b0-ae16-7aafb3b89053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:53:59.265606 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246917d2-a218-42b0-ae16-7aafb3b89053 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:53:59.265893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246917d2-a218-42b0-ae16-7aafb3b89053 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:53:59.266173 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246917d2-a218-42b0-ae16-7aafb3b89053 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:53:59.266981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246917d2-a218-42b0-ae16-7aafb3b89053 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:53:59.267291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246917d2-a218-42b0-ae16-7aafb3b89053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:53:59.268702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246917d2-a218-42b0-ae16-7aafb3b89053 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:53:59.278343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 243/480] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:53:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:00.328088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599daed-25f2-46b9-bec2-410a83e49965 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:00.329564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599daed-25f2-46b9-bec2-410a83e49965 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:00.330352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599daed-25f2-46b9-bec2-410a83e49965 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:00.332390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 238/481] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:00.338179 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f59120-6cb7-4122-834e-4a7628236aaf None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:00.339084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f59120-6cb7-4122-834e-4a7628236aaf None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:00.339615 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f59120-6cb7-4122-834e-4a7628236aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:00.361129 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-97f59120-6cb7-4122-834e-4a7628236aaf None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:00.362651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-97f59120-6cb7-4122-834e-4a7628236aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:00.452978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 244/482] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:00.476723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599daed-25f2-46b9-bec2-410a83e49965 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:00.486222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377120c6-29b0-41e0-97b9-92ef014167ea None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:00.486842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377120c6-29b0-41e0-97b9-92ef014167ea None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:00.487321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377120c6-29b0-41e0-97b9-92ef014167ea None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:00.489232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 245/483] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:54:00.494812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc79e3c-afd5-4623-9a3b-f16c8de6dac7 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:00.495425 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc79e3c-afd5-4623-9a3b-f16c8de6dac7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:00.495955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc79e3c-afd5-4623-9a3b-f16c8de6dac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:00.514342 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2cc79e3c-afd5-4623-9a3b-f16c8de6dac7 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:00.515912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2cc79e3c-afd5-4623-9a3b-f16c8de6dac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:00.644968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599daed-25f2-46b9-bec2-410a83e49965 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:00.660587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 246/484] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:54:00.690865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b7a10-e2cf-40d2-acb3-9312cb691b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:00.691693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b7a10-e2cf-40d2-acb3-9312cb691b34 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:00.691975 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b7a10-e2cf-40d2-acb3-9312cb691b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:00.692250 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b7a10-e2cf-40d2-acb3-9312cb691b34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:00.709300 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b7a10-e2cf-40d2-acb3-9312cb691b34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-8a4b7a10-e2cf-40d2-acb3-9312cb691b34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:00.716971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:00.717839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:00.718626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 239/485] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:00.724355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc79e3c-afd5-4623-9a3b-f16c8de6dac7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:00.736010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b7a10-e2cf-40d2-acb3-9312cb691b34 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:00.771622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:00.775636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:00.929288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc79e3c-afd5-4623-9a3b-f16c8de6dac7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:00.970812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b7a10-e2cf-40d2-acb3-9312cb691b34 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:01.016220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722ee03-da4c-422b-872e-91092547a4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:01.017040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722ee03-da4c-422b-872e-91092547a4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.017316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722ee03-da4c-422b-872e-91092547a4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.017589 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722ee03-da4c-422b-872e-91092547a4c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.039000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722ee03-da4c-422b-872e-91092547a4c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:01.047356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c926d8-aece-4b77-9e95-0b6321f57cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:01.048241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c926d8-aece-4b77-9e95-0b6321f57cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.048529 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c926d8-aece-4b77-9e95-0b6321f57cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.048805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c926d8-aece-4b77-9e95-0b6321f57cf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-2722ee03-da4c-422b-872e-91092547a4c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:01.049426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:01.050271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:01.051042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:01.051042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:01.051042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:01.051042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:01.051042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:01.051042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:01.051042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:01.051042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:01.051042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:01.051042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:01.051042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:01.051042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:01.051663 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c926d8-aece-4b77-9e95-0b6321f57cf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:01.051663 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c926d8-aece-4b77-9e95-0b6321f57cf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:01.051663 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c926d8-aece-4b77-9e95-0b6321f57cf1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:01.054498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 247/486] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:01.067031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 240/487] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:01.068668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722ee03-da4c-422b-872e-91092547a4c3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:01.072856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c926d8-aece-4b77-9e95-0b6321f57cf1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:01.165693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:01.172256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:01.185078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722ee03-da4c-422b-872e-91092547a4c3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:01.205038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c926d8-aece-4b77-9e95-0b6321f57cf1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:01.227970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee112f-f026-47dd-9836-e23fe392101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:01.229049 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee112f-f026-47dd-9836-e23fe392101c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.229554 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee112f-f026-47dd-9836-e23fe392101c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.230030 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee112f-f026-47dd-9836-e23fe392101c None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.230994 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee112f-f026-47dd-9836-e23fe392101c None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:01.231721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee112f-f026-47dd-9836-e23fe392101c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:01.233435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee112f-f026-47dd-9836-e23fe392101c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:01.244412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 248/488] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:01.245604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a5547-658c-4449-ab86-29bcc13e255a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:01.246408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a5547-658c-4449-ab86-29bcc13e255a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.246686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a5547-658c-4449-ab86-29bcc13e255a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.246975 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a5547-658c-4449-ab86-29bcc13e255a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.247212 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23f35-62d2-4461-879f-3bbe9fc5f759 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.247892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23f35-62d2-4461-879f-3bbe9fc5f759 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.248431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23f35-62d2-4461-879f-3bbe9fc5f759 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.250433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 249/489] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:01.251338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b23f35-62d2-4461-879f-3bbe9fc5f759 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:01.263777 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a5547-658c-4449-ab86-29bcc13e255a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:01.265628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a5547-658c-4449-ab86-29bcc13e255a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-971a5547-658c-4449-ab86-29bcc13e255a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:01.288825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:01.289767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:01.290611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:01.290611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:01.290611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:01.290611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:01.290611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 241/490] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:01.291596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4ebd1-012a-4929-91c0-0359ba3b25ea None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.291884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4ebd1-012a-4929-91c0-0359ba3b25ea None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.292162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4ebd1-012a-4929-91c0-0359ba3b25ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.313060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-1de4ebd1-012a-4929-91c0-0359ba3b25ea None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:01.314275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1de4ebd1-012a-4929-91c0-0359ba3b25ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:01.375619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b23f35-62d2-4461-879f-3bbe9fc5f759 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:01.403491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 242/491] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:01.405084 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4ebd1-012a-4929-91c0-0359ba3b25ea None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:01.435852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec5eeb-05f3-4fbd-b8e8-8806d4b27b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:01.437021 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec5eeb-05f3-4fbd-b8e8-8806d4b27b0f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.437503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec5eeb-05f3-4fbd-b8e8-8806d4b27b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.438109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec5eeb-05f3-4fbd-b8e8-8806d4b27b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.450320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec5eeb-05f3-4fbd-b8e8-8806d4b27b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:01.452273 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec5eeb-05f3-4fbd-b8e8-8806d4b27b0f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:01.462489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-77ec5eeb-05f3-4fbd-b8e8-8806d4b27b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:01.475186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:01.476115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:01.476959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:01.476959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:01.476959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:01.476959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:01.479487 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 250/492] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:01.480545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec5eeb-05f3-4fbd-b8e8-8806d4b27b0f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:01.486510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:01.541215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4ebd1-012a-4929-91c0-0359ba3b25ea None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:01.582561 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8cc59-c22c-45eb-9626-146d8f86b2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:01.584229 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8cc59-c22c-45eb-9626-146d8f86b2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.584735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8cc59-c22c-45eb-9626-146d8f86b2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.585313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8cc59-c22c-45eb-9626-146d8f86b2c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.586404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8cc59-c22c-45eb-9626-146d8f86b2c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:01.586911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8cc59-c22c-45eb-9626-146d8f86b2c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:01.588685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8cc59-c22c-45eb-9626-146d8f86b2c0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:01.602637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 243/493] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:01.604010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8cc59-c22c-45eb-9626-146d8f86b2c0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:01.607759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec5eeb-05f3-4fbd-b8e8-8806d4b27b0f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:01.651876 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71caa3-2ff2-47e8-a172-eb30e283de10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:01.652297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71caa3-2ff2-47e8-a172-eb30e283de10 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.652584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71caa3-2ff2-47e8-a172-eb30e283de10 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.652880 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71caa3-2ff2-47e8-a172-eb30e283de10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.667205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71caa3-2ff2-47e8-a172-eb30e283de10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-7d71caa3-2ff2-47e8-a172-eb30e283de10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:01.677798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:01.678770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:01.679583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:01.679583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:01.679583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:01.679583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:01.679583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:01.679583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:01.679583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:01.679583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:01.679583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:01.679583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:01.679583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:01.679583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:01.683460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 251/494] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:01.684584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71caa3-2ff2-47e8-a172-eb30e283de10 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:01.733870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8cc59-c22c-45eb-9626-146d8f86b2c0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:01.757539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:01.760066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:01.778644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffdd1a-7011-4ed9-8659-d954ddec4b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:01.779322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffdd1a-7011-4ed9-8659-d954ddec4b2e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.779636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffdd1a-7011-4ed9-8659-d954ddec4b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.779924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffdd1a-7011-4ed9-8659-d954ddec4b2e None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.780645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffdd1a-7011-4ed9-8659-d954ddec4b2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:01.780953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffdd1a-7011-4ed9-8659-d954ddec4b2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:01.782333 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffdd1a-7011-4ed9-8659-d954ddec4b2e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:01.799415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 244/495] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:01.799415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffdd1a-7011-4ed9-8659-d954ddec4b2e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:01.813260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71caa3-2ff2-47e8-a172-eb30e283de10 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:01.854165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe4395-5492-472f-a19c-df3c9a2a26f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:01.855447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe4395-5492-472f-a19c-df3c9a2a26f9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.855447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe4395-5492-472f-a19c-df3c9a2a26f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.855742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe4395-5492-472f-a19c-df3c9a2a26f9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.868746 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe4395-5492-472f-a19c-df3c9a2a26f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:01.870234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe4395-5492-472f-a19c-df3c9a2a26f9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-45fe4395-5492-472f-a19c-df3c9a2a26f9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:01.886137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:01.887098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:01.888014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:01.888014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:01.888014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:01.888014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:01.888014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:01.888014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:01.888014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:01.888014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:01.888014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:54:01.888014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:01.892007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 252/496] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:01.893427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcef66b-baf6-4c4c-8147-b164b870d50f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.893722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcef66b-baf6-4c4c-8147-b164b870d50f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.894008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcef66b-baf6-4c4c-8147-b164b870d50f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.895500 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 253/497] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:01.896601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcef66b-baf6-4c4c-8147-b164b870d50f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:01.917581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffdd1a-7011-4ed9-8659-d954ddec4b2e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:01.961902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f962c-5e0a-414d-aec8-40aa93d0d8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:01.962636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f962c-5e0a-414d-aec8-40aa93d0d8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:01.962886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f962c-5e0a-414d-aec8-40aa93d0d8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:01.963153 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f962c-5e0a-414d-aec8-40aa93d0d8d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:01.964092 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f962c-5e0a-414d-aec8-40aa93d0d8d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:01.964291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f962c-5e0a-414d-aec8-40aa93d0d8d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:01.965651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f962c-5e0a-414d-aec8-40aa93d0d8d3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:01.978823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 245/498] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:01.978823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f962c-5e0a-414d-aec8-40aa93d0d8d3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:02.010455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcef66b-baf6-4c4c-8147-b164b870d50f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:02.023713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:02.031537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:02.046111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e55c6-7118-487b-8a1f-eae9093c0f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:02.047029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e55c6-7118-487b-8a1f-eae9093c0f3c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:02.047406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e55c6-7118-487b-8a1f-eae9093c0f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:02.047716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e55c6-7118-487b-8a1f-eae9093c0f3c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:02.055254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e55c6-7118-487b-8a1f-eae9093c0f3c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:02.056758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e55c6-7118-487b-8a1f-eae9093c0f3c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c95e55c6-7118-487b-8a1f-eae9093c0f3c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:02.072496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:02.073651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:02.074914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:02.074914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:02.074914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:02.074914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:02.074914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:02.074914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:02.074914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:02.074914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:02.074914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:54:02.074914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:02.074914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 254/499] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:02.075634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67024f6-fd04-492c-809a-8ddf99e70b46 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:02.075634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67024f6-fd04-492c-809a-8ddf99e70b46 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:02.075773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67024f6-fd04-492c-809a-8ddf99e70b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:02.092024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f962c-5e0a-414d-aec8-40aa93d0d8d3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:02.092826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c67024f6-fd04-492c-809a-8ddf99e70b46 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:02.094039 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c67024f6-fd04-492c-809a-8ddf99e70b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:02.126315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce632da9-5710-49a2-837d-9599ba3c0925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:02.126985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce632da9-5710-49a2-837d-9599ba3c0925 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:02.127269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce632da9-5710-49a2-837d-9599ba3c0925 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:02.127578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce632da9-5710-49a2-837d-9599ba3c0925 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:02.128414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce632da9-5710-49a2-837d-9599ba3c0925 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:02.128725 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce632da9-5710-49a2-837d-9599ba3c0925 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:02.130790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce632da9-5710-49a2-837d-9599ba3c0925 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:02.160750 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 255/500] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:02.161613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67024f6-fd04-492c-809a-8ddf99e70b46 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:02.171155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 246/501] 10.222.0.75 () {56 vars in 1137 bytes} [Thu Jul 14 10:54:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:02.172551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c73d09-c042-4cbd-b6de-baea7e943dd4 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:02.172896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c73d09-c042-4cbd-b6de-baea7e943dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:02.173261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c73d09-c042-4cbd-b6de-baea7e943dd4 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:02.174844 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 247/502] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:02.175920 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c73d09-c042-4cbd-b6de-baea7e943dd4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:02.275794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67024f6-fd04-492c-809a-8ddf99e70b46 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:02.279157 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:02.286547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c73d09-c042-4cbd-b6de-baea7e943dd4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:02.286993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:02.312742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffcbe1-46c4-4c86-a55b-473bd98b9e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:02.313560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffcbe1-46c4-4c86-a55b-473bd98b9e84 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:02.313850 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffcbe1-46c4-4c86-a55b-473bd98b9e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:02.315520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffcbe1-46c4-4c86-a55b-473bd98b9e84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:02.326671 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffcbe1-46c4-4c86-a55b-473bd98b9e84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:02.328191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffcbe1-46c4-4c86-a55b-473bd98b9e84 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:02.328534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b221d45-d50a-4b7a-8d98-94dd859ed998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:02.329213 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b221d45-d50a-4b7a-8d98-94dd859ed998 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:02.329490 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b221d45-d50a-4b7a-8d98-94dd859ed998 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:02.329770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b221d45-d50a-4b7a-8d98-94dd859ed998 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:02.330601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b221d45-d50a-4b7a-8d98-94dd859ed998 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:02.330905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b221d45-d50a-4b7a-8d98-94dd859ed998 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:02.332389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b221d45-d50a-4b7a-8d98-94dd859ed998 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c9ffcbe1-46c4-4c86-a55b-473bd98b9e84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:02.350997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:02.352174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:02.353188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:02.353188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:02.353188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:02.353188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:02.353188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 256/503] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:02.353188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffcbe1-46c4-4c86-a55b-473bd98b9e84 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:02.354289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 248/504] 10.222.0.75 () {56 vars in 1137 bytes} [Thu Jul 14 10:54:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:02.355293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3741c81-bc23-4fe0-8e80-311c04e73b7e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:02.355602 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3741c81-bc23-4fe0-8e80-311c04e73b7e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:02.355878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3741c81-bc23-4fe0-8e80-311c04e73b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:02.372650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e3741c81-bc23-4fe0-8e80-311c04e73b7e None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:02.373870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e3741c81-bc23-4fe0-8e80-311c04e73b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:02.447876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 249/505] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:02.448720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3741c81-bc23-4fe0-8e80-311c04e73b7e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:02.464042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffcbe1-46c4-4c86-a55b-473bd98b9e84 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:02.500838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcd481-3c92-463a-8d10-7e9de5b3c8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:02.501503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcd481-3c92-463a-8d10-7e9de5b3c8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:02.501784 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcd481-3c92-463a-8d10-7e9de5b3c8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:02.502059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcd481-3c92-463a-8d10-7e9de5b3c8e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:02.533241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:02.543466 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:02.566460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3741c81-bc23-4fe0-8e80-311c04e73b7e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:02.574572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcd481-3c92-463a-8d10-7e9de5b3c8e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:02.576078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcd481-3c92-463a-8d10-7e9de5b3c8e5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:02.602074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549bc5c-0366-4e47-afe2-91e8eb6a067c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:02.602873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549bc5c-0366-4e47-afe2-91e8eb6a067c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:02.603156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549bc5c-0366-4e47-afe2-91e8eb6a067c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:02.603571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549bc5c-0366-4e47-afe2-91e8eb6a067c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:02.613798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549bc5c-0366-4e47-afe2-91e8eb6a067c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-1549bc5c-0366-4e47-afe2-91e8eb6a067c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:02.622127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:02.623102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:02.624054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 250/506] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:02.624767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549bc5c-0366-4e47-afe2-91e8eb6a067c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:02.691734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 257/507] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:54:02.693314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcd481-3c92-463a-8d10-7e9de5b3c8e5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:02.745764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549bc5c-0366-4e47-afe2-91e8eb6a067c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:02.759323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:02.783446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021087c-869e-4f4a-8c57-37d568ee5681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:02.784018 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021087c-869e-4f4a-8c57-37d568ee5681 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:02.784304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021087c-869e-4f4a-8c57-37d568ee5681 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:02.784582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021087c-869e-4f4a-8c57-37d568ee5681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:02.799648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcd481-3c92-463a-8d10-7e9de5b3c8e5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:02.800085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:02.836068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811540c-8e8c-40a1-a1bb-cb0305d923d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:02.836743 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811540c-8e8c-40a1-a1bb-cb0305d923d8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:02.837029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811540c-8e8c-40a1-a1bb-cb0305d923d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:02.837310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811540c-8e8c-40a1-a1bb-cb0305d923d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:02.839181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811540c-8e8c-40a1-a1bb-cb0305d923d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:02.839181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811540c-8e8c-40a1-a1bb-cb0305d923d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:02.839790 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811540c-8e8c-40a1-a1bb-cb0305d923d8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:02.852734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 258/508] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:02.853853 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811540c-8e8c-40a1-a1bb-cb0305d923d8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:02.856000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021087c-869e-4f4a-8c57-37d568ee5681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:02.857490 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021087c-869e-4f4a-8c57-37d568ee5681 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:02.957205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811540c-8e8c-40a1-a1bb-cb0305d923d8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:02.962430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 251/509] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:54:02.963485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021087c-869e-4f4a-8c57-37d568ee5681 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:02.981584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:02.997331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f960978-cf6c-4b12-9f5d-53d83518a869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:02.997331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f960978-cf6c-4b12-9f5d-53d83518a869 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:02.997638 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f960978-cf6c-4b12-9f5d-53d83518a869 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:02.997721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f960978-cf6c-4b12-9f5d-53d83518a869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:03.008525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f960978-cf6c-4b12-9f5d-53d83518a869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:03.009999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f960978-cf6c-4b12-9f5d-53d83518a869 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:03.015109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-9f960978-cf6c-4b12-9f5d-53d83518a869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:03.028187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:03.029092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:03.029951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:03.029951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:03.029951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:03.029951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:03.029951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 259/510] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:03.030206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f960978-cf6c-4b12-9f5d-53d83518a869 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:03.084616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021087c-869e-4f4a-8c57-37d568ee5681 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:03.120856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a23acc-f97d-4d81-acf7-d0a3a0ec8076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:03.121532 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a23acc-f97d-4d81-acf7-d0a3a0ec8076 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:03.121820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a23acc-f97d-4d81-acf7-d0a3a0ec8076 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:03.122101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a23acc-f97d-4d81-acf7-d0a3a0ec8076 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:03.122853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a23acc-f97d-4d81-acf7-d0a3a0ec8076 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:03.123174 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a23acc-f97d-4d81-acf7-d0a3a0ec8076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:03.124602 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a23acc-f97d-4d81-acf7-d0a3a0ec8076 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:03.133570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 252/511] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:03.134525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a23acc-f97d-4d81-acf7-d0a3a0ec8076 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:03.135832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f960978-cf6c-4b12-9f5d-53d83518a869 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:03.173573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d61d2-0eaa-4c15-969a-82ba77971bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:03.174384 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d61d2-0eaa-4c15-969a-82ba77971bac None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:03.174669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d61d2-0eaa-4c15-969a-82ba77971bac None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:03.174949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d61d2-0eaa-4c15-969a-82ba77971bac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:03.186269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d61d2-0eaa-4c15-969a-82ba77971bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:03.187816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d61d2-0eaa-4c15-969a-82ba77971bac None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-4d4d61d2-0eaa-4c15-969a-82ba77971bac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:03.210920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:03.212040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:03.213000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:03.213000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:03.213000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:03.213000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:03.213000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 260/512] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:03.213331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d61d2-0eaa-4c15-969a-82ba77971bac None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:03.243449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:03.244714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a23acc-f97d-4d81-acf7-d0a3a0ec8076 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:03.251411 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:03.281710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b4567-2916-4bad-8cbe-d4c019a3f49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:03.282678 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b4567-2916-4bad-8cbe-d4c019a3f49c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:03.283012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b4567-2916-4bad-8cbe-d4c019a3f49c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:03.283290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b4567-2916-4bad-8cbe-d4c019a3f49c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:03.291125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b4567-2916-4bad-8cbe-d4c019a3f49c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:03.292628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b4567-2916-4bad-8cbe-d4c019a3f49c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-147b4567-2916-4bad-8cbe-d4c019a3f49c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:03.308795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:03.309965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:03.311012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:03.311012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:03.311012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:03.311012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:03.311012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:03.311012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:03.311012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:03.311012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:03.311012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:54:03.311012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:03.311012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 253/513] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Jul 14 10:54:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:03.311012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b4567-2916-4bad-8cbe-d4c019a3f49c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:03.325594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d61d2-0eaa-4c15-969a-82ba77971bac None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:03.361718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9ff88-7ed2-458c-a772-5430a150e6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:03.362431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9ff88-7ed2-458c-a772-5430a150e6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:03.362720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9ff88-7ed2-458c-a772-5430a150e6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:03.362990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9ff88-7ed2-458c-a772-5430a150e6f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:03.373598 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9ff88-7ed2-458c-a772-5430a150e6f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:03.375064 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9ff88-7ed2-458c-a772-5430a150e6f8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-d2e9ff88-7ed2-458c-a772-5430a150e6f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:03.392071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:03.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:03.393971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:03.393971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:03.393971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:03.393971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:03.393971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 261/514] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:03.394227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9ff88-7ed2-458c-a772-5430a150e6f8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:03.417531 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b4567-2916-4bad-8cbe-d4c019a3f49c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:03.452005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ce366-55c6-4b28-94f0-3255957e892a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:03.452983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ce366-55c6-4b28-94f0-3255957e892a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:03.453272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ce366-55c6-4b28-94f0-3255957e892a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:03.453557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ce366-55c6-4b28-94f0-3255957e892a None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:03.454287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494ce366-55c6-4b28-94f0-3255957e892a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:03.454598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494ce366-55c6-4b28-94f0-3255957e892a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:03.456033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494ce366-55c6-4b28-94f0-3255957e892a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:03.468206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 254/515] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:03.469218 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ce366-55c6-4b28-94f0-3255957e892a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:03.486625 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:03.501172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:03.510973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9ff88-7ed2-458c-a772-5430a150e6f8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:03.549156 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddb46a-7753-497e-9a0c-c052779b9643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:03.549963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddb46a-7753-497e-9a0c-c052779b9643 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:03.550254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddb46a-7753-497e-9a0c-c052779b9643 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:03.550537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddb46a-7753-497e-9a0c-c052779b9643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:03.564053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddb46a-7753-497e-9a0c-c052779b9643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:03.565537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddb46a-7753-497e-9a0c-c052779b9643 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:03.578418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ce366-55c6-4b28-94f0-3255957e892a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-e8ddb46a-7753-497e-9a0c-c052779b9643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:03.584498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:03.585405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:03.586235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:03.586235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:03.586235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:03.586235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:03.586235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 262/516] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:03.586712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddb46a-7753-497e-9a0c-c052779b9643 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:03.617380 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee3e11-da1a-4612-9883-ecfa2cdf9ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:03.617968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee3e11-da1a-4612-9883-ecfa2cdf9ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:03.618253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee3e11-da1a-4612-9883-ecfa2cdf9ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:03.618531 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee3e11-da1a-4612-9883-ecfa2cdf9ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:03.619249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee3e11-da1a-4612-9883-ecfa2cdf9ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:03.619590 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee3e11-da1a-4612-9883-ecfa2cdf9ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:03.620989 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee3e11-da1a-4612-9883-ecfa2cdf9ce3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:03.633437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 255/517] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:03.634268 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee3e11-da1a-4612-9883-ecfa2cdf9ce3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:03.695176 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddb46a-7753-497e-9a0c-c052779b9643 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:03.723833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:03.727748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:03.734441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37017219-29ed-44d0-a4a5-448acdfaf3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:03.735118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37017219-29ed-44d0-a4a5-448acdfaf3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:03.735422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37017219-29ed-44d0-a4a5-448acdfaf3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:03.735709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37017219-29ed-44d0-a4a5-448acdfaf3d9 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:03.736550 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37017219-29ed-44d0-a4a5-448acdfaf3d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:03.736863 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37017219-29ed-44d0-a4a5-448acdfaf3d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:03.738308 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37017219-29ed-44d0-a4a5-448acdfaf3d9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:03.739970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee3e11-da1a-4612-9883-ecfa2cdf9ce3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:03.776818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48107c30-68b2-4061-8e94-e94508ddcb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:03.777484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48107c30-68b2-4061-8e94-e94508ddcb96 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:03.782290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48107c30-68b2-4061-8e94-e94508ddcb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:03.782465 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 263/518] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Jul 14 10:54:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:03.782547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48107c30-68b2-4061-8e94-e94508ddcb96 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:03.782547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48107c30-68b2-4061-8e94-e94508ddcb96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:03.782665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37017219-29ed-44d0-a4a5-448acdfaf3d9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:03.782730 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48107c30-68b2-4061-8e94-e94508ddcb96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:03.782730 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48107c30-68b2-4061-8e94-e94508ddcb96 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:03.795165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 256/519] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:03.796023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48107c30-68b2-4061-8e94-e94508ddcb96 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:03.890769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37017219-29ed-44d0-a4a5-448acdfaf3d9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:03.902926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48107c30-68b2-4061-8e94-e94508ddcb96 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:03.928094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc318b25-daa7-4f3c-bd2f-afc05280fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:03.928770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc318b25-daa7-4f3c-bd2f-afc05280fa1d None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:03.929053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc318b25-daa7-4f3c-bd2f-afc05280fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:03.929329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc318b25-daa7-4f3c-bd2f-afc05280fa1d None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:03.930036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc318b25-daa7-4f3c-bd2f-afc05280fa1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:03.931521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc318b25-daa7-4f3c-bd2f-afc05280fa1d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:03.941007 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c92b8-7945-4310-b418-8bd9735f61f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:03.941691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c92b8-7945-4310-b418-8bd9735f61f5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:03.941961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c92b8-7945-4310-b418-8bd9735f61f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:03.942242 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c92b8-7945-4310-b418-8bd9735f61f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:03.943608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c92b8-7945-4310-b418-8bd9735f61f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:03.944008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c92b8-7945-4310-b418-8bd9735f61f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:03.945573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c92b8-7945-4310-b418-8bd9735f61f5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:03.953393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 264/520] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:54:03 2022] POST /identity/v3/users => generated 247 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:54:03.954217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc318b25-daa7-4f3c-bd2f-afc05280fa1d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:03.968696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 257/521] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:03.969804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c92b8-7945-4310-b418-8bd9735f61f5 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:03.982732 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:04.033457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:04.106547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc318b25-daa7-4f3c-bd2f-afc05280fa1d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:04.120708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c92b8-7945-4310-b418-8bd9735f61f5 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:04.157456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d2863-5cbf-4922-8735-7fe25081300d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:04.157456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d2863-5cbf-4922-8735-7fe25081300d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:04.157456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d2863-5cbf-4922-8735-7fe25081300d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:04.157788 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d2863-5cbf-4922-8735-7fe25081300d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:04.166789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d2863-5cbf-4922-8735-7fe25081300d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:04.168307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d2863-5cbf-4922-8735-7fe25081300d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:04.170508 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd795d-72d1-4062-be7a-87adaa651664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:04.171168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd795d-72d1-4062-be7a-87adaa651664 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:04.171512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd795d-72d1-4062-be7a-87adaa651664 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:04.171800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd795d-72d1-4062-be7a-87adaa651664 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-4a1d2863-5cbf-4922-8735-7fe25081300d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:04.186093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:04.186988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:04.187927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:04.187927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:04.187927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:04.187927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:04.187927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:04.187927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:04.187927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:04.187927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:04.187927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 265/522] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:04.188569 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d2863-5cbf-4922-8735-7fe25081300d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:04.257138 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd795d-72d1-4062-be7a-87adaa651664 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:04.258632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd795d-72d1-4062-be7a-87adaa651664 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:04.267365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:04.312517 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d2863-5cbf-4922-8735-7fe25081300d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:04.315118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:04.356100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78872f4-b848-497f-869c-16393d000820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:04.356916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78872f4-b848-497f-869c-16393d000820 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:04.357206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78872f4-b848-497f-869c-16393d000820 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:04.357483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78872f4-b848-497f-869c-16393d000820 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:04.371948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78872f4-b848-497f-869c-16393d000820 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:04.373718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78872f4-b848-497f-869c-16393d000820 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:04.387259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 258/523] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:54:04.388217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd795d-72d1-4062-be7a-87adaa651664 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-e78872f4-b848-497f-869c-16393d000820 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:04.391167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:04.392050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:04.392891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:04.392891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:04.392891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:04.392891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:04.392891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:04.392891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:04.392891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:04.392891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:04.393404 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 266/524] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:04.393829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78872f4-b848-497f-869c-16393d000820 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:04.509543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78872f4-b848-497f-869c-16393d000820 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:04.516675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd795d-72d1-4062-be7a-87adaa651664 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:04.524363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:04.546456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb4962-557a-4cf7-b8de-c91fa10e0cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:04.547125 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb4962-557a-4cf7-b8de-c91fa10e0cfb None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:04.547427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb4962-557a-4cf7-b8de-c91fa10e0cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:04.547708 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb4962-557a-4cf7-b8de-c91fa10e0cfb None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:04.548493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb4962-557a-4cf7-b8de-c91fa10e0cfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:04.548794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb4962-557a-4cf7-b8de-c91fa10e0cfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:04.550710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb4962-557a-4cf7-b8de-c91fa10e0cfb None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:04.569035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b26244-5ab2-47c4-81b6-ad6526ea1dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:04.569832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b26244-5ab2-47c4-81b6-ad6526ea1dad None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:04.570108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b26244-5ab2-47c4-81b6-ad6526ea1dad None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:04.570378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b26244-5ab2-47c4-81b6-ad6526ea1dad None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:04.577609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 267/525] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:04.578794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb4962-557a-4cf7-b8de-c91fa10e0cfb None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:04.579013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b26244-5ab2-47c4-81b6-ad6526ea1dad None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:04.580442 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b26244-5ab2-47c4-81b6-ad6526ea1dad None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-a4b26244-5ab2-47c4-81b6-ad6526ea1dad None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:04.598616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:04.600173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:04.601034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:04.601034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:04.601034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:04.601034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:04.601034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:04.601034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:04.601034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:04.601034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:04.601034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:54:04.601034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:04.605374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 259/526] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:04.606241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b26244-5ab2-47c4-81b6-ad6526ea1dad None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:04.615169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:04.709889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb4962-557a-4cf7-b8de-c91fa10e0cfb None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:04.731110 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b26244-5ab2-47c4-81b6-ad6526ea1dad None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:04.748675 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757709-737c-4409-ab1c-7a063fcca0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:04.749727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757709-737c-4409-ab1c-7a063fcca0fb None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:04.750160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757709-737c-4409-ab1c-7a063fcca0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:04.750585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757709-737c-4409-ab1c-7a063fcca0fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:04.765679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2757709-737c-4409-ab1c-7a063fcca0fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:04.767074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2757709-737c-4409-ab1c-7a063fcca0fb None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:04.776455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc3d18-3364-4084-b570-9f5be756d04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:04.777397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc3d18-3364-4084-b570-9f5be756d04c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:04.777894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc3d18-3364-4084-b570-9f5be756d04c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:04.778003 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc3d18-3364-4084-b570-9f5be756d04c None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:04.778750 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc3d18-3364-4084-b570-9f5be756d04c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:04.779061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc3d18-3364-4084-b570-9f5be756d04c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:04.780563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc3d18-3364-4084-b570-9f5be756d04c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-d2757709-737c-4409-ab1c-7a063fcca0fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:04.788282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:04.789159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:04.789990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:04.789990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:04.789990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:04.789990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:04.791712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 268/527] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:04.793398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757709-737c-4409-ab1c-7a063fcca0fb None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:04.800842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 260/528] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:04.801685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc3d18-3364-4084-b570-9f5be756d04c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:04.838793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:04.890334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:04.917156 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757709-737c-4409-ab1c-7a063fcca0fb None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:04.926927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc3d18-3364-4084-b570-9f5be756d04c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:04.955469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8527f22-1c4a-4504-8b2b-c380e4f5b0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:04.956133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8527f22-1c4a-4504-8b2b-c380e4f5b0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:04.956410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8527f22-1c4a-4504-8b2b-c380e4f5b0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:04.956684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8527f22-1c4a-4504-8b2b-c380e4f5b0d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:04.957506 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8527f22-1c4a-4504-8b2b-c380e4f5b0d3 None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=d1ad1ffe4d99420cb568d6154f779838) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:04.957907 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8527f22-1c4a-4504-8b2b-c380e4f5b0d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:04.959975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8527f22-1c4a-4504-8b2b-c380e4f5b0d3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:04.964391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa266109-e319-4556-aff4-50365bf92b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:04.965101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa266109-e319-4556-aff4-50365bf92b37 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:04.965395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa266109-e319-4556-aff4-50365bf92b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:04.965668 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa266109-e319-4556-aff4-50365bf92b37 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:04.966509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa266109-e319-4556-aff4-50365bf92b37 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:04.966821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa266109-e319-4556-aff4-50365bf92b37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:04.975561 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 269/529] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:04 2022] GET /identity/v3/role_assignments?user.id=d1ad1ffe4d99420cb568d6154f779838&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:04.975801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa266109-e319-4556-aff4-50365bf92b37 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:04.975884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8527f22-1c4a-4504-8b2b-c380e4f5b0d3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:04.995368 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 261/530] 10.222.0.75 () {56 vars in 1139 bytes} [Thu Jul 14 10:54:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:04.996301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa266109-e319-4556-aff4-50365bf92b37 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:05.110222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8527f22-1c4a-4504-8b2b-c380e4f5b0d3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:05.123192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:05.149092 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa266109-e319-4556-aff4-50365bf92b37 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:05.159416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391935f-b17f-4821-ad87-656f63558e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:05.159416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391935f-b17f-4821-ad87-656f63558e60 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:05.159416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391935f-b17f-4821-ad87-656f63558e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:05.159416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391935f-b17f-4821-ad87-656f63558e60 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:05.159845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391935f-b17f-4821-ad87-656f63558e60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:05.159845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391935f-b17f-4821-ad87-656f63558e60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:05.161398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391935f-b17f-4821-ad87-656f63558e60 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:05.172223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:05.180620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 270/531] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:05.180620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391935f-b17f-4821-ad87-656f63558e60 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:05.201941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cf083-1470-4730-84a5-3186de43498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:05.202665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cf083-1470-4730-84a5-3186de43498f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:05.203249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cf083-1470-4730-84a5-3186de43498f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:05.203556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cf083-1470-4730-84a5-3186de43498f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:05.204402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5cf083-1470-4730-84a5-3186de43498f None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=80230b1b73944e8ebb6fec3dbe0d5c4b) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:05.204716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5cf083-1470-4730-84a5-3186de43498f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:05.206170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5cf083-1470-4730-84a5-3186de43498f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:05.215475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 262/532] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:04 2022] GET /identity/v3/role_assignments?user.id=80230b1b73944e8ebb6fec3dbe0d5c4b&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:05.216688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cf083-1470-4730-84a5-3186de43498f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:05.379077 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391935f-b17f-4821-ad87-656f63558e60 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:05.440780 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cf083-1470-4730-84a5-3186de43498f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:05.442666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15af5d-9562-41b9-907e-b11b5bb0305a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:05.444124 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15af5d-9562-41b9-907e-b11b5bb0305a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:05.444510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15af5d-9562-41b9-907e-b11b5bb0305a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:05.444856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15af5d-9562-41b9-907e-b11b5bb0305a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:05.508774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f52a4-aa15-48d3-9b5f-0e426a4dd7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:05.509599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f52a4-aa15-48d3-9b5f-0e426a4dd7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:05.509898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f52a4-aa15-48d3-9b5f-0e426a4dd7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:05.510192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f52a4-aa15-48d3-9b5f-0e426a4dd7a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:05.523946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:05.527215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15af5d-9562-41b9-907e-b11b5bb0305a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:05.529460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15af5d-9562-41b9-907e-b11b5bb0305a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:05.536327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f52a4-aa15-48d3-9b5f-0e426a4dd7a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:05.537846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f52a4-aa15-48d3-9b5f-0e426a4dd7a4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-5a5f52a4-aa15-48d3-9b5f-0e426a4dd7a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:05.573621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:05.574530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:05.575403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:05.575403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:05.575403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:05.575403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:05.575403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:05.575403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:05.575403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:05.575403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:05.575403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 263/533] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:05.584674 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f52a4-aa15-48d3-9b5f-0e426a4dd7a4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:05.613061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:05.721211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 271/534] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:54:05.732504 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15af5d-9562-41b9-907e-b11b5bb0305a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:05.822313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f52a4-aa15-48d3-9b5f-0e426a4dd7a4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:05.877114 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96264aa6-df80-4fe0-9b78-acf004565b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:05.878271 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96264aa6-df80-4fe0-9b78-acf004565b66 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:05.878908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96264aa6-df80-4fe0-9b78-acf004565b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:05.879412 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96264aa6-df80-4fe0-9b78-acf004565b66 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:05.880899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96264aa6-df80-4fe0-9b78-acf004565b66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:05.881680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96264aa6-df80-4fe0-9b78-acf004565b66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:05.883358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96264aa6-df80-4fe0-9b78-acf004565b66 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:05.891279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:05.898863 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 264/535] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:05.903139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15af5d-9562-41b9-907e-b11b5bb0305a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:05.905371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96264aa6-df80-4fe0-9b78-acf004565b66 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:05.967590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaedf3d6-c370-4a3d-94fe-a45120759e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:05.968854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedf3d6-c370-4a3d-94fe-a45120759e83 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:05.969334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedf3d6-c370-4a3d-94fe-a45120759e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:05.969824 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedf3d6-c370-4a3d-94fe-a45120759e83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:05.992612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaedf3d6-c370-4a3d-94fe-a45120759e83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:05.994051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaedf3d6-c370-4a3d-94fe-a45120759e83 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:06.013222 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-aaedf3d6-c370-4a3d-94fe-a45120759e83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:06.030941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:06.031885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:06.032723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:06.032723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:06.032723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:06.032723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:06.042446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 272/536] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:06.043479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaedf3d6-c370-4a3d-94fe-a45120759e83 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:06.096867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96264aa6-df80-4fe0-9b78-acf004565b66 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:06.143260 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae9cbf-2da5-42e0-8fdc-efc7b6a8225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:06.143985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae9cbf-2da5-42e0-8fdc-efc7b6a8225c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:06.144273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae9cbf-2da5-42e0-8fdc-efc7b6a8225c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:06.144672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae9cbf-2da5-42e0-8fdc-efc7b6a8225c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:06.146300 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae9cbf-2da5-42e0-8fdc-efc7b6a8225c None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=7e572bf785284a8693cf386ced004524) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:06.146851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae9cbf-2da5-42e0-8fdc-efc7b6a8225c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:06.148956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae9cbf-2da5-42e0-8fdc-efc7b6a8225c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:06.155830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 265/537] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:05 2022] GET /identity/v3/role_assignments?user.id=7e572bf785284a8693cf386ced004524&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:06.242108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaedf3d6-c370-4a3d-94fe-a45120759e83 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:06.249197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1eb89-a3b6-4fe2-b160-53c5f7657395 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:06.249397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1eb89-a3b6-4fe2-b160-53c5f7657395 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:06.249673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1eb89-a3b6-4fe2-b160-53c5f7657395 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:06.251215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 266/538] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:06.259656 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c40a3-7b1c-4a82-a39d-2a67fac95496 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:06.260041 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c40a3-7b1c-4a82-a39d-2a67fac95496 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:06.260332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c40a3-7b1c-4a82-a39d-2a67fac95496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:06.296293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-398c40a3-7b1c-4a82-a39d-2a67fac95496 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:06.297544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-398c40a3-7b1c-4a82-a39d-2a67fac95496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:06.301282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8745db-1cef-4ed4-8e03-40d7a937411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:06.302018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8745db-1cef-4ed4-8e03-40d7a937411f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:06.302304 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8745db-1cef-4ed4-8e03-40d7a937411f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:06.302582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8745db-1cef-4ed4-8e03-40d7a937411f None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:06.303332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8745db-1cef-4ed4-8e03-40d7a937411f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:06.303667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8745db-1cef-4ed4-8e03-40d7a937411f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:06.306517 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8745db-1cef-4ed4-8e03-40d7a937411f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:06.336602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 273/539] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:06.342019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8745db-1cef-4ed4-8e03-40d7a937411f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:06.382889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:06.470267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 267/540] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:06.496101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8745db-1cef-4ed4-8e03-40d7a937411f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:06.510781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c40a3-7b1c-4a82-a39d-2a67fac95496 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:06.542980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2516f1-7591-492e-9950-268b5959ced9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:06.544507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2516f1-7591-492e-9950-268b5959ced9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:06.546083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2516f1-7591-492e-9950-268b5959ced9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:06.546652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2516f1-7591-492e-9950-268b5959ced9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:06.563711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2516f1-7591-492e-9950-268b5959ced9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:06.568786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2516f1-7591-492e-9950-268b5959ced9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-de2516f1-7591-492e-9950-268b5959ced9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:06.605198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:06.606089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:06.606916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:06.606916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:06.606916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:06.606916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:06.606916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:06.606916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:06.606916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:06.606916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:06.611212 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 274/541] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:06.614953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2516f1-7591-492e-9950-268b5959ced9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:06.640943 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:06.698848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c40a3-7b1c-4a82-a39d-2a67fac95496 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:06.733555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:06.737802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce186e6-d48a-40f0-a4fa-337978491ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:06.738861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce186e6-d48a-40f0-a4fa-337978491ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:06.739473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce186e6-d48a-40f0-a4fa-337978491ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:06.739949 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce186e6-d48a-40f0-a4fa-337978491ca9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:06.751525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce186e6-d48a-40f0-a4fa-337978491ca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-dce186e6-d48a-40f0-a4fa-337978491ca9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:06.760230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:06.761142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:06.761934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:06.761934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:06.761934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:06.761934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:06.761934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:06.761934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:06.761934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:06.761934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:06.761934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:06.761934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:06.761934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:06.761934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:06.764611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 268/542] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:06.771361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce186e6-d48a-40f0-a4fa-337978491ca9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:06.784601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2516f1-7591-492e-9950-268b5959ced9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:06.832816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72296cd-9e66-4584-bb84-6b856d1535a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:06.834082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72296cd-9e66-4584-bb84-6b856d1535a7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:06.834374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72296cd-9e66-4584-bb84-6b856d1535a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:06.834664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72296cd-9e66-4584-bb84-6b856d1535a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:06.835684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72296cd-9e66-4584-bb84-6b856d1535a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:06.835893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72296cd-9e66-4584-bb84-6b856d1535a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:06.837536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72296cd-9e66-4584-bb84-6b856d1535a7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:06.858628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 275/543] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:06.864122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72296cd-9e66-4584-bb84-6b856d1535a7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:06.981436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce186e6-d48a-40f0-a4fa-337978491ca9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:06.998515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72296cd-9e66-4584-bb84-6b856d1535a7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:07.039850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:07.043903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:07.047522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d50d6-3a25-4194-a8ac-64f438c35b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:07.048219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d50d6-3a25-4194-a8ac-64f438c35b27 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:07.048501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d50d6-3a25-4194-a8ac-64f438c35b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:07.048774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d50d6-3a25-4194-a8ac-64f438c35b27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:07.050377 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d50d6-3a25-4194-a8ac-64f438c35b27 None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=dadd485b7cac42f4aa079cab91973c35) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:07.050690 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d50d6-3a25-4194-a8ac-64f438c35b27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:07.052711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d50d6-3a25-4194-a8ac-64f438c35b27 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:07.062074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a64aae-59e4-4d53-8460-dbce085c35fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:07.062703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a64aae-59e4-4d53-8460-dbce085c35fb None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:07.062985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a64aae-59e4-4d53-8460-dbce085c35fb None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:07.063262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a64aae-59e4-4d53-8460-dbce085c35fb None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:07.064038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a64aae-59e4-4d53-8460-dbce085c35fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:07.064339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a64aae-59e4-4d53-8460-dbce085c35fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:07.065713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a64aae-59e4-4d53-8460-dbce085c35fb None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:07.072106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 276/544] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:06 2022] GET /identity/v3/role_assignments?user.id=dadd485b7cac42f4aa079cab91973c35&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:07.087112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 269/545] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:07.092942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d50d6-3a25-4194-a8ac-64f438c35b27 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:07.225837 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d50d6-3a25-4194-a8ac-64f438c35b27 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:07.283093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b5f3f-3b15-4386-9cf1-aba0eff64843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:07.283945 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b5f3f-3b15-4386-9cf1-aba0eff64843 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:07.284230 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b5f3f-3b15-4386-9cf1-aba0eff64843 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:07.284512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b5f3f-3b15-4386-9cf1-aba0eff64843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:07.293223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c62c1-6a15-4b9b-8d57-ee2e70493a0a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:07.293824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c62c1-6a15-4b9b-8d57-ee2e70493a0a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:07.294302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c62c1-6a15-4b9b-8d57-ee2e70493a0a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:07.296253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 270/546] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:07.299806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b5f3f-3b15-4386-9cf1-aba0eff64843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:07.301583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b5f3f-3b15-4386-9cf1-aba0eff64843 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:07.303792 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108177e-fb69-43da-90b3-75356c1da1a8 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:07.304304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108177e-fb69-43da-90b3-75356c1da1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:07.304792 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108177e-fb69-43da-90b3-75356c1da1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-671b5f3f-3b15-4386-9cf1-aba0eff64843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:07.321147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:07.322032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:07.322875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:07.322875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:07.322875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:07.322875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:07.322875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 277/547] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:07.331415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b5f3f-3b15-4386-9cf1-aba0eff64843 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:07.337902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a108177e-fb69-43da-90b3-75356c1da1a8 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:07.339866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a108177e-fb69-43da-90b3-75356c1da1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:07.395812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:07.447893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b5f3f-3b15-4386-9cf1-aba0eff64843 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:07.486908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ba036-be2e-4bef-9eff-b18118382fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:07.487627 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ba036-be2e-4bef-9eff-b18118382fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:07.487918 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ba036-be2e-4bef-9eff-b18118382fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:07.488202 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ba036-be2e-4bef-9eff-b18118382fb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:07.488941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ba036-be2e-4bef-9eff-b18118382fb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:07.489300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ba036-be2e-4bef-9eff-b18118382fb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:07.491199 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ba036-be2e-4bef-9eff-b18118382fb2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:07.505800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 278/548] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:07.507160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efabb68-4653-4315-8366-3dbf4d4ecb63 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:07.511589 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efabb68-4653-4315-8366-3dbf4d4ecb63 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:07.511930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efabb68-4653-4315-8366-3dbf4d4ecb63 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:07.513313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 279/549] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:07.515293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efabb68-4653-4315-8366-3dbf4d4ecb63 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:07.540331 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 271/550] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:07.541746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7b4bf-5f03-47b1-bc0b-2dac6e3b2c86 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:07.542648 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7b4bf-5f03-47b1-bc0b-2dac6e3b2c86 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:07.542948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7b4bf-5f03-47b1-bc0b-2dac6e3b2c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:07.563263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-69d7b4bf-5f03-47b1-bc0b-2dac6e3b2c86 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:07.564426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-69d7b4bf-5f03-47b1-bc0b-2dac6e3b2c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:07.648759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 272/551] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:07.649645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7b4bf-5f03-47b1-bc0b-2dac6e3b2c86 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:07.667351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efabb68-4653-4315-8366-3dbf4d4ecb63 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:07.711761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315cd82-292d-4472-bfea-6d005d22a8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:07.712994 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315cd82-292d-4472-bfea-6d005d22a8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:07.713485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315cd82-292d-4472-bfea-6d005d22a8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:07.714047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315cd82-292d-4472-bfea-6d005d22a8c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:07.721246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:07.724427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315cd82-292d-4472-bfea-6d005d22a8c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:07.727237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315cd82-292d-4472-bfea-6d005d22a8c8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:07.743626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c315cd82-292d-4472-bfea-6d005d22a8c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:07.745146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:07.746180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:07.747065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:07.747065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:07.747065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:07.747065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:07.747065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:07.747065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:07.747065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:07.747065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:07.747065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:54:07.747065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:07.748933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 280/552] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:54:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:07.754412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315cd82-292d-4472-bfea-6d005d22a8c8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:07.798383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7b4bf-5f03-47b1-bc0b-2dac6e3b2c86 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:07.843714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447427f-f969-4cec-ad9e-4a454a3d5950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:07.844451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447427f-f969-4cec-ad9e-4a454a3d5950 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:07.844761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447427f-f969-4cec-ad9e-4a454a3d5950 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:07.845545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447427f-f969-4cec-ad9e-4a454a3d5950 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:07.860075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447427f-f969-4cec-ad9e-4a454a3d5950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-6447427f-f969-4cec-ad9e-4a454a3d5950 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:07.870130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:07.871070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:07.872083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 273/553] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:07.872790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447427f-f969-4cec-ad9e-4a454a3d5950 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:07.890949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315cd82-292d-4472-bfea-6d005d22a8c8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:07.940246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9f8a1-28e6-4b10-9fce-f1df1acf4cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:07.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9f8a1-28e6-4b10-9fce-f1df1acf4cde None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:07.941671 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9f8a1-28e6-4b10-9fce-f1df1acf4cde None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:07.941937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9f8a1-28e6-4b10-9fce-f1df1acf4cde None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:07.956898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9f8a1-28e6-4b10-9fce-f1df1acf4cde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-4aa9f8a1-28e6-4b10-9fce-f1df1acf4cde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:07.968264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:07.969260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:07.970161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:07.970161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:07.970161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:07.970161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:07.970161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:07.970161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:07.970161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:07.970161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:07.970161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:07.970161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:07.970161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:07.970161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:07.974424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 281/554] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:07.975336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9f8a1-28e6-4b10-9fce-f1df1acf4cde None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:08.004416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447427f-f969-4cec-ad9e-4a454a3d5950 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:08.007807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:08.012704 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:08.047034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d5ad3-2818-4962-a664-feacfe0ff13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:08.048224 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d5ad3-2818-4962-a664-feacfe0ff13d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:08.048715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d5ad3-2818-4962-a664-feacfe0ff13d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:08.049192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d5ad3-2818-4962-a664-feacfe0ff13d None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:08.050235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d5ad3-2818-4962-a664-feacfe0ff13d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:08.050835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d5ad3-2818-4962-a664-feacfe0ff13d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:08.052760 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d5ad3-2818-4962-a664-feacfe0ff13d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:08.078914 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 274/555] 10.222.0.75 () {56 vars in 1135 bytes} [Thu Jul 14 10:54:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:08.080297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d5ad3-2818-4962-a664-feacfe0ff13d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:08.105084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9f8a1-28e6-4b10-9fce-f1df1acf4cde None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:08.147874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb035a8-bd39-45fa-8afe-5895fc5d5098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:08.148785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb035a8-bd39-45fa-8afe-5895fc5d5098 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:08.149095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb035a8-bd39-45fa-8afe-5895fc5d5098 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:08.149375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb035a8-bd39-45fa-8afe-5895fc5d5098 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:08.150125 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb035a8-bd39-45fa-8afe-5895fc5d5098 None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:08.150576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb035a8-bd39-45fa-8afe-5895fc5d5098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:08.152222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb035a8-bd39-45fa-8afe-5895fc5d5098 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:08.165076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 282/556] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:08.165916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb035a8-bd39-45fa-8afe-5895fc5d5098 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:08.208977 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d5ad3-2818-4962-a664-feacfe0ff13d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:08.245024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893f6e7-7c2c-4959-a32b-7ca2aa02b351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:08.246145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893f6e7-7c2c-4959-a32b-7ca2aa02b351 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:08.246628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893f6e7-7c2c-4959-a32b-7ca2aa02b351 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:08.247120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893f6e7-7c2c-4959-a32b-7ca2aa02b351 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:08.248118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893f6e7-7c2c-4959-a32b-7ca2aa02b351 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:08.248646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893f6e7-7c2c-4959-a32b-7ca2aa02b351 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:08.250428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893f6e7-7c2c-4959-a32b-7ca2aa02b351 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:08.265797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 275/557] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:08.267013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893f6e7-7c2c-4959-a32b-7ca2aa02b351 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:08.276494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:08.288081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb035a8-bd39-45fa-8afe-5895fc5d5098 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:08.299850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:08.334946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac6fc9-3db8-4c20-8bff-520381fc34a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:08.335645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac6fc9-3db8-4c20-8bff-520381fc34a0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:08.335981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac6fc9-3db8-4c20-8bff-520381fc34a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:08.336256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac6fc9-3db8-4c20-8bff-520381fc34a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:08.398466 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893f6e7-7c2c-4959-a32b-7ca2aa02b351 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:08.443689 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116b5bb-69cd-4c95-8c40-a77d04195360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:08.443898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac6fc9-3db8-4c20-8bff-520381fc34a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:08.444955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116b5bb-69cd-4c95-8c40-a77d04195360 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:08.445274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac6fc9-3db8-4c20-8bff-520381fc34a0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:08.445698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116b5bb-69cd-4c95-8c40-a77d04195360 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:08.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116b5bb-69cd-4c95-8c40-a77d04195360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:08.459567 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116b5bb-69cd-4c95-8c40-a77d04195360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:08.461482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116b5bb-69cd-4c95-8c40-a77d04195360 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-6116b5bb-69cd-4c95-8c40-a77d04195360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:08.481160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:08.482140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:08.483011 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:08.483011 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:08.483011 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:08.483011 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:08.485485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 276/558] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:08.486772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116b5bb-69cd-4c95-8c40-a77d04195360 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:08.550791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:08.567442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 283/559] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:54:08.567442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac6fc9-3db8-4c20-8bff-520381fc34a0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:08.596630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:08.629735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116b5bb-69cd-4c95-8c40-a77d04195360 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:08.671859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab888a-393d-42ee-83a2-38b2e07ea17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:08.673813 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab888a-393d-42ee-83a2-38b2e07ea17f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:08.674310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab888a-393d-42ee-83a2-38b2e07ea17f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:08.674793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab888a-393d-42ee-83a2-38b2e07ea17f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:08.688473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab888a-393d-42ee-83a2-38b2e07ea17f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:08.689988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac6fc9-3db8-4c20-8bff-520381fc34a0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:08.690890 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab888a-393d-42ee-83a2-38b2e07ea17f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-01ab888a-393d-42ee-83a2-38b2e07ea17f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:08.717040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:08.718266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:08.719351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:08.719351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:08.719351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:08.719351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:08.719351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 277/560] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:08.719351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab888a-393d-42ee-83a2-38b2e07ea17f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:08.747660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a68e9-4425-4098-b7b5-b30f02eafd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:08.749364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a68e9-4425-4098-b7b5-b30f02eafd0a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:08.749364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a68e9-4425-4098-b7b5-b30f02eafd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:08.749364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a68e9-4425-4098-b7b5-b30f02eafd0a None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:08.749613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a68e9-4425-4098-b7b5-b30f02eafd0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:08.749896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a68e9-4425-4098-b7b5-b30f02eafd0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:08.751270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a68e9-4425-4098-b7b5-b30f02eafd0a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:08.764713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 284/561] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:08.766130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da129c3-5ba5-4918-99cf-b383eb22a9d7 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:08.766428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da129c3-5ba5-4918-99cf-b383eb22a9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:08.766712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da129c3-5ba5-4918-99cf-b383eb22a9d7 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:08.768166 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 285/562] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:08.772923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da129c3-5ba5-4918-99cf-b383eb22a9d7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:08.833014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:08.843163 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab888a-393d-42ee-83a2-38b2e07ea17f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:08.875896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:08.886514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee113be-16f9-4222-a910-594a43f62c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:08.887409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee113be-16f9-4222-a910-594a43f62c46 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:08.887765 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee113be-16f9-4222-a910-594a43f62c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:08.888133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee113be-16f9-4222-a910-594a43f62c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:08.900729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee113be-16f9-4222-a910-594a43f62c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:08.901581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da129c3-5ba5-4918-99cf-b383eb22a9d7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:08.902169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee113be-16f9-4222-a910-594a43f62c46 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-5ee113be-16f9-4222-a910-594a43f62c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:08.924235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:08.925410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:08.926408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:08.926408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:08.926408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:08.926408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:08.930313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 278/563] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:08.931169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee113be-16f9-4222-a910-594a43f62c46 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:08.945584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30f9a6-8f16-488c-a3d2-c01f18c73ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:08.946381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30f9a6-8f16-488c-a3d2-c01f18c73ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:08.946669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30f9a6-8f16-488c-a3d2-c01f18c73ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:08.946948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30f9a6-8f16-488c-a3d2-c01f18c73ae2 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:08.947710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30f9a6-8f16-488c-a3d2-c01f18c73ae2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:08.948022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30f9a6-8f16-488c-a3d2-c01f18c73ae2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:08.949397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30f9a6-8f16-488c-a3d2-c01f18c73ae2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:08.967624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 286/564] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:08.967624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89b49-4a3c-4bb4-a0e1-ef6bcc7ec7c1 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:08.967624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89b49-4a3c-4bb4-a0e1-ef6bcc7ec7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:08.967624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89b49-4a3c-4bb4-a0e1-ef6bcc7ec7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:08.983257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-14b89b49-4a3c-4bb4-a0e1-ef6bcc7ec7c1 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:08.984498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-14b89b49-4a3c-4bb4-a0e1-ef6bcc7ec7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:09.055335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee113be-16f9-4222-a910-594a43f62c46 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:09.071442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 287/565] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:09.071442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b89b49-4a3c-4bb4-a0e1-ef6bcc7ec7c1 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:09.108545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64237dce-758e-4404-903f-483ba886781b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:09.109459 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64237dce-758e-4404-903f-483ba886781b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:09.109749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64237dce-758e-4404-903f-483ba886781b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:09.110031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64237dce-758e-4404-903f-483ba886781b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:09.117879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64237dce-758e-4404-903f-483ba886781b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:09.119361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64237dce-758e-4404-903f-483ba886781b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:09.123933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-64237dce-758e-4404-903f-483ba886781b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:09.135057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:09.136102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:09.137027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:09.137027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:09.137027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:09.137027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:09.137027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:09.137027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:09.137027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:09.137027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:09.137027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:54:09.137027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:09.137027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 279/566] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:09.137623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64237dce-758e-4404-903f-483ba886781b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:09.183920 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:09.199249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b89b49-4a3c-4bb4-a0e1-ef6bcc7ec7c1 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:09.236037 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3bfdc-d5ae-433a-9d76-aaae74d9f166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:09.237335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3bfdc-d5ae-433a-9d76-aaae74d9f166 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:09.237805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3bfdc-d5ae-433a-9d76-aaae74d9f166 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:09.238091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3bfdc-d5ae-433a-9d76-aaae74d9f166 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:09.238834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3bfdc-d5ae-433a-9d76-aaae74d9f166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:09.239148 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3bfdc-d5ae-433a-9d76-aaae74d9f166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:09.241104 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3bfdc-d5ae-433a-9d76-aaae74d9f166 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:09.252671 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 288/567] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:09.253477 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3bfdc-d5ae-433a-9d76-aaae74d9f166 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:09.261628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64237dce-758e-4404-903f-483ba886781b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:09.301475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1420b-dfd3-48df-98ce-2184dbe78e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:09.302794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1420b-dfd3-48df-98ce-2184dbe78e10 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:09.303459 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1420b-dfd3-48df-98ce-2184dbe78e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:09.303752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1420b-dfd3-48df-98ce-2184dbe78e10 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:09.312822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1420b-dfd3-48df-98ce-2184dbe78e10 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:09.314286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1420b-dfd3-48df-98ce-2184dbe78e10 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-56a1420b-dfd3-48df-98ce-2184dbe78e10 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:09.329902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:09.330868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:09.331835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:09.331835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:09.331835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:09.331835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:09.331835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:09.331835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:09.331835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:09.331835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:09.331835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:54:09.331835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:09.331835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 280/568] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:09.332409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1420b-dfd3-48df-98ce-2184dbe78e10 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:09.372197 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:09.375231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3bfdc-d5ae-433a-9d76-aaae74d9f166 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:09.409602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb7867-3b57-45bd-8be9-24ab5d2a2f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:09.410403 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb7867-3b57-45bd-8be9-24ab5d2a2f65 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:09.410691 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb7867-3b57-45bd-8be9-24ab5d2a2f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:09.410977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb7867-3b57-45bd-8be9-24ab5d2a2f65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:09.421064 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb7867-3b57-45bd-8be9-24ab5d2a2f65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b7cb7867-3b57-45bd-8be9-24ab5d2a2f65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:09.429312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:09.430239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:09.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 289/569] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:09.431786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb7867-3b57-45bd-8be9-24ab5d2a2f65 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:09.433398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:09.436956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1420b-dfd3-48df-98ce-2184dbe78e10 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:09.470875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2b43d-0148-40de-89d7-064c5f6594a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:09.471569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2b43d-0148-40de-89d7-064c5f6594a6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:09.471863 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2b43d-0148-40de-89d7-064c5f6594a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:09.472133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2b43d-0148-40de-89d7-064c5f6594a6 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:09.472961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2b43d-0148-40de-89d7-064c5f6594a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:09.473268 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2b43d-0148-40de-89d7-064c5f6594a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:09.475259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2b43d-0148-40de-89d7-064c5f6594a6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:09.492763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 281/570] 10.222.0.75 () {56 vars in 1137 bytes} [Thu Jul 14 10:54:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:09.493544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2b43d-0148-40de-89d7-064c5f6594a6 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:09.535638 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb7867-3b57-45bd-8be9-24ab5d2a2f65 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:09.569949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50137c-b9bd-4f71-b743-d9a6b53de089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:09.570754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50137c-b9bd-4f71-b743-d9a6b53de089 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:09.571047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50137c-b9bd-4f71-b743-d9a6b53de089 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:09.571332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50137c-b9bd-4f71-b743-d9a6b53de089 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:09.578098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50137c-b9bd-4f71-b743-d9a6b53de089 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:09.579575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50137c-b9bd-4f71-b743-d9a6b53de089 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:09.589539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2b43d-0148-40de-89d7-064c5f6594a6 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-9a50137c-b9bd-4f71-b743-d9a6b53de089 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:09.593646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:09.594770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:09.595905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:09.595905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:09.595905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:09.595905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:09.595905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:09.595905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:09.595905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:09.595905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:09.595905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 290/571] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:09.596399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda1d34-5fe1-4762-9c99-285f924840ad None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:09.596643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda1d34-5fe1-4762-9c99-285f924840ad None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:09.596928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda1d34-5fe1-4762-9c99-285f924840ad None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:09.599720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 291/572] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:09.599720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda1d34-5fe1-4762-9c99-285f924840ad None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:09.621475 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:09.629593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e863e00-3dbc-4262-817e-906ecb34c460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:09.630293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e863e00-3dbc-4262-817e-906ecb34c460 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:09.630577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e863e00-3dbc-4262-817e-906ecb34c460 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:09.630859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e863e00-3dbc-4262-817e-906ecb34c460 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:09.631719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e863e00-3dbc-4262-817e-906ecb34c460 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:09.632031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e863e00-3dbc-4262-817e-906ecb34c460 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:09.633478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e863e00-3dbc-4262-817e-906ecb34c460 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:09.650651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 282/573] 10.222.0.75 () {56 vars in 1137 bytes} [Thu Jul 14 10:54:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:09.651552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e863e00-3dbc-4262-817e-906ecb34c460 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:09.668768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:09.706119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda1d34-5fe1-4762-9c99-285f924840ad None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:09.740010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05471fb7-888c-43fa-8a9c-5a503b13f59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:09.740671 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05471fb7-888c-43fa-8a9c-5a503b13f59b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:09.741094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05471fb7-888c-43fa-8a9c-5a503b13f59b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:09.741375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05471fb7-888c-43fa-8a9c-5a503b13f59b None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:09.742115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05471fb7-888c-43fa-8a9c-5a503b13f59b None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:09.742424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05471fb7-888c-43fa-8a9c-5a503b13f59b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:09.743812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05471fb7-888c-43fa-8a9c-5a503b13f59b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:09.748707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e863e00-3dbc-4262-817e-906ecb34c460 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:09.751499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 292/574] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:09.752280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05471fb7-888c-43fa-8a9c-5a503b13f59b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:09.782518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf71c08-99b5-443c-b7bb-fc45e000cf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:09.783312 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf71c08-99b5-443c-b7bb-fc45e000cf8a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:09.783627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf71c08-99b5-443c-b7bb-fc45e000cf8a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:09.783913 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf71c08-99b5-443c-b7bb-fc45e000cf8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:09.794270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf71c08-99b5-443c-b7bb-fc45e000cf8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:09.795753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf71c08-99b5-443c-b7bb-fc45e000cf8a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-abf71c08-99b5-443c-b7bb-fc45e000cf8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:09.813645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:09.814507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:09.815688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:09.815688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:09.815688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:09.815688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:09.815688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 283/575] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:09.816321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09038e3c-dd3a-4bee-ac3e-7e4ef4ac399b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:09.816607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09038e3c-dd3a-4bee-ac3e-7e4ef4ac399b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:09.816881 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09038e3c-dd3a-4bee-ac3e-7e4ef4ac399b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:09.833679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-09038e3c-dd3a-4bee-ac3e-7e4ef4ac399b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:09.834873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-09038e3c-dd3a-4bee-ac3e-7e4ef4ac399b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:09.836424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:09.854459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05471fb7-888c-43fa-8a9c-5a503b13f59b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:09.888245 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d200f3-6f1b-4fe7-bada-fe334b0e616e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:09.888910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d200f3-6f1b-4fe7-bada-fe334b0e616e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:09.889196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d200f3-6f1b-4fe7-bada-fe334b0e616e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:09.889479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d200f3-6f1b-4fe7-bada-fe334b0e616e None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:09.890265 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d200f3-6f1b-4fe7-bada-fe334b0e616e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:09.890579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d200f3-6f1b-4fe7-bada-fe334b0e616e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:09.892049 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d200f3-6f1b-4fe7-bada-fe334b0e616e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:09.895883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:09.911422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 284/576] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:09.911422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09038e3c-dd3a-4bee-ac3e-7e4ef4ac399b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:09.911636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 293/577] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:09.911636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d200f3-6f1b-4fe7-bada-fe334b0e616e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:10.010014 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09038e3c-dd3a-4bee-ac3e-7e4ef4ac399b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:10.013074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d200f3-6f1b-4fe7-bada-fe334b0e616e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:10.043073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d0b0e-a417-4b63-a286-6af8bbf05889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:10.043880 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d0b0e-a417-4b63-a286-6af8bbf05889 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:10.044160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d0b0e-a417-4b63-a286-6af8bbf05889 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:10.044436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d0b0e-a417-4b63-a286-6af8bbf05889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:10.048080 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7e8db-4c28-4bd1-9b2c-6d176d63538b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:10.048871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7e8db-4c28-4bd1-9b2c-6d176d63538b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:10.049160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7e8db-4c28-4bd1-9b2c-6d176d63538b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:10.049439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7e8db-4c28-4bd1-9b2c-6d176d63538b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:10.054595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d0b0e-a417-4b63-a286-6af8bbf05889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:10.056085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d0b0e-a417-4b63-a286-6af8bbf05889 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:10.060081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7e8db-4c28-4bd1-9b2c-6d176d63538b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:10.061529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7e8db-4c28-4bd1-9b2c-6d176d63538b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:10.063208 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-110d0b0e-a417-4b63-a286-6af8bbf05889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:10.075234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:10.076252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:10.077116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:10.077116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:10.077116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:10.077116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:10.077116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 285/578] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:10.077394 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d0b0e-a417-4b63-a286-6af8bbf05889 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-34d7e8db-4c28-4bd1-9b2c-6d176d63538b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:10.080536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:10.081428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:10.082302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:10.082302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:10.082302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:10.082302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:10.082302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 294/579] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:10.082961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7e8db-4c28-4bd1-9b2c-6d176d63538b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:10.132328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:10.181031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d0b0e-a417-4b63-a286-6af8bbf05889 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:10.183524 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7e8db-4c28-4bd1-9b2c-6d176d63538b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:10.217047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23b4fb-9c09-44f9-8ac4-371b019521a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:10.217712 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23b4fb-9c09-44f9-8ac4-371b019521a5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:10.217998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23b4fb-9c09-44f9-8ac4-371b019521a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:10.218278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23b4fb-9c09-44f9-8ac4-371b019521a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:10.218481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f413a1-7e72-48ad-8429-854545c156c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:10.219001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23b4fb-9c09-44f9-8ac4-371b019521a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:10.219135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f413a1-7e72-48ad-8429-854545c156c1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:10.219308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23b4fb-9c09-44f9-8ac4-371b019521a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:10.219447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f413a1-7e72-48ad-8429-854545c156c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:10.219728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f413a1-7e72-48ad-8429-854545c156c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:10.220551 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f413a1-7e72-48ad-8429-854545c156c1 None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=978e71dfbb134f41871114824b03fbeb) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:10.220732 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23b4fb-9c09-44f9-8ac4-371b019521a5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:10.220866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f413a1-7e72-48ad-8429-854545c156c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:10.222288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f413a1-7e72-48ad-8429-854545c156c1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:10.228324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 295/580] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:10 2022] GET /identity/v3/role_assignments?user.id=978e71dfbb134f41871114824b03fbeb&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:10.229097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f413a1-7e72-48ad-8429-854545c156c1 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:10.244707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 286/581] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:10.245532 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23b4fb-9c09-44f9-8ac4-371b019521a5 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:10.303587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:10.359980 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23b4fb-9c09-44f9-8ac4-371b019521a5 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:10.360765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f413a1-7e72-48ad-8429-854545c156c1 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:10.371759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:10.400642 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb4da3-3a2b-49a7-b392-d65f9e3f3ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:10.401544 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb4da3-3a2b-49a7-b392-d65f9e3f3ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:10.401841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb4da3-3a2b-49a7-b392-d65f9e3f3ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:10.402276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb4da3-3a2b-49a7-b392-d65f9e3f3ea8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:10.409595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6f5f0-9bb5-43d9-bbe9-3e7139b9866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:10.410274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6f5f0-9bb5-43d9-bbe9-3e7139b9866b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:10.410566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6f5f0-9bb5-43d9-bbe9-3e7139b9866b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:10.410853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6f5f0-9bb5-43d9-bbe9-3e7139b9866b None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:10.412434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6f5f0-9bb5-43d9-bbe9-3e7139b9866b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:10.412606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6f5f0-9bb5-43d9-bbe9-3e7139b9866b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:10.413908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6f5f0-9bb5-43d9-bbe9-3e7139b9866b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:10.415794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb4da3-3a2b-49a7-b392-d65f9e3f3ea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-53bb4da3-3a2b-49a7-b392-d65f9e3f3ea8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:10.423584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:10.424728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 296/582] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:10.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb4da3-3a2b-49a7-b392-d65f9e3f3ea8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:10.432985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 287/583] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:10.433994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6f5f0-9bb5-43d9-bbe9-3e7139b9866b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:10.541476 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb4da3-3a2b-49a7-b392-d65f9e3f3ea8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:10.541941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6f5f0-9bb5-43d9-bbe9-3e7139b9866b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:10.561488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:10.578658 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9691175-d2be-425d-b78e-13ad5ff7de76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:10.579579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9691175-d2be-425d-b78e-13ad5ff7de76 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:10.579865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9691175-d2be-425d-b78e-13ad5ff7de76 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:10.580144 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9691175-d2be-425d-b78e-13ad5ff7de76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:10.581522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226079a-2c10-4a36-ad9a-50297c0209cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:10.582188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226079a-2c10-4a36-ad9a-50297c0209cd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:10.582475 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226079a-2c10-4a36-ad9a-50297c0209cd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:10.582757 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226079a-2c10-4a36-ad9a-50297c0209cd None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:10.584090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226079a-2c10-4a36-ad9a-50297c0209cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:10.584399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226079a-2c10-4a36-ad9a-50297c0209cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:10.585775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226079a-2c10-4a36-ad9a-50297c0209cd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:10.587861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9691175-d2be-425d-b78e-13ad5ff7de76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:10.589749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9691175-d2be-425d-b78e-13ad5ff7de76 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:10.597815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 297/584] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:10.602813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226079a-2c10-4a36-ad9a-50297c0209cd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f9691175-d2be-425d-b78e-13ad5ff7de76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:10.609024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:10.609940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:10.610836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:10.610836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:10.610836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:10.610836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:10.610836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:10.610836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:10.610836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:10.610836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:10.610836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 288/585] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:10.611803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9691175-d2be-425d-b78e-13ad5ff7de76 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:10.645737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:10.717552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226079a-2c10-4a36-ad9a-50297c0209cd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:10.720327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9691175-d2be-425d-b78e-13ad5ff7de76 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:10.756396 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633414b3-1d87-4c81-9ca8-7000ea1dcb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:10.757220 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633414b3-1d87-4c81-9ca8-7000ea1dcb4a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:10.757497 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633414b3-1d87-4c81-9ca8-7000ea1dcb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:10.757663 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777a843-d550-4ea9-a6f2-2bf629b0631a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:10.757786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633414b3-1d87-4c81-9ca8-7000ea1dcb4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:10.758423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777a843-d550-4ea9-a6f2-2bf629b0631a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:10.758804 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777a843-d550-4ea9-a6f2-2bf629b0631a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:10.759091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777a843-d550-4ea9-a6f2-2bf629b0631a None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:10.760139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777a843-d550-4ea9-a6f2-2bf629b0631a None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:10.760457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777a843-d550-4ea9-a6f2-2bf629b0631a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:10.761968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777a843-d550-4ea9-a6f2-2bf629b0631a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:10.765419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633414b3-1d87-4c81-9ca8-7000ea1dcb4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:10.766874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633414b3-1d87-4c81-9ca8-7000ea1dcb4a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:10.775629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 298/586] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:10.775629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777a843-d550-4ea9-a6f2-2bf629b0631a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-633414b3-1d87-4c81-9ca8-7000ea1dcb4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:10.784564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:10.785501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:10.791453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:10.791453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:10.791453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:10.791453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:10.791453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:10.791453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:10.791453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:10.791453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:10.791453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 289/587] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:10.791453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633414b3-1d87-4c81-9ca8-7000ea1dcb4a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:10.822005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:10.886666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777a843-d550-4ea9-a6f2-2bf629b0631a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:10.891467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:10.903292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633414b3-1d87-4c81-9ca8-7000ea1dcb4a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:10.921754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47656b7-4fe6-4004-97bc-212ac4b7b0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:10.922555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47656b7-4fe6-4004-97bc-212ac4b7b0ef None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:10.922841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47656b7-4fe6-4004-97bc-212ac4b7b0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:10.923121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47656b7-4fe6-4004-97bc-212ac4b7b0ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:10.930909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47656b7-4fe6-4004-97bc-212ac4b7b0ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:10.932387 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47656b7-4fe6-4004-97bc-212ac4b7b0ef None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:10.939987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a444b-ebcd-40c2-bda1-aa20e9ea8976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:10.940649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a444b-ebcd-40c2-bda1-aa20e9ea8976 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:10.940933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a444b-ebcd-40c2-bda1-aa20e9ea8976 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:10.941209 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a444b-ebcd-40c2-bda1-aa20e9ea8976 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:10.941987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a444b-ebcd-40c2-bda1-aa20e9ea8976 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:10.942343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a444b-ebcd-40c2-bda1-aa20e9ea8976 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:10.944307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a444b-ebcd-40c2-bda1-aa20e9ea8976 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b47656b7-4fe6-4004-97bc-212ac4b7b0ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:10.947832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:10.948733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:10.949721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:10.949721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:10.949721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:10.949721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:10.949721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:10.949721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:10.949721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:10.949721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:10.949721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:54:10.949721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:10.949721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 299/588] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Jul 14 10:54:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:10.950312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47656b7-4fe6-4004-97bc-212ac4b7b0ef None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:10.966023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 290/589] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:10.966911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a444b-ebcd-40c2-bda1-aa20e9ea8976 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:11.057675 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:11.062123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47656b7-4fe6-4004-97bc-212ac4b7b0ef None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:11.069149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a444b-ebcd-40c2-bda1-aa20e9ea8976 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:11.097774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16eef0-cdf5-4a06-8629-1eae7a7150d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:11.098495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16eef0-cdf5-4a06-8629-1eae7a7150d5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:11.099303 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16eef0-cdf5-4a06-8629-1eae7a7150d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:11.099612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16eef0-cdf5-4a06-8629-1eae7a7150d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:11.103481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625add7-fe2d-41cb-ae2e-45da4b39ead7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:11.104143 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625add7-fe2d-41cb-ae2e-45da4b39ead7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:11.104431 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625add7-fe2d-41cb-ae2e-45da4b39ead7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:11.104712 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625add7-fe2d-41cb-ae2e-45da4b39ead7 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:11.105496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625add7-fe2d-41cb-ae2e-45da4b39ead7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:11.105808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625add7-fe2d-41cb-ae2e-45da4b39ead7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:11.107261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625add7-fe2d-41cb-ae2e-45da4b39ead7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:11.109765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16eef0-cdf5-4a06-8629-1eae7a7150d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:11.111188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16eef0-cdf5-4a06-8629-1eae7a7150d5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:11.124397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 291/590] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:11.124397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625add7-fe2d-41cb-ae2e-45da4b39ead7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:11.134293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-ce16eef0-cdf5-4a06-8629-1eae7a7150d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:11.135774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:11.136939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:11.137805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:11.137805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:11.137805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:11.137805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:11.139778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 300/591] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:11.139778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16eef0-cdf5-4a06-8629-1eae7a7150d5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:11.235454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625add7-fe2d-41cb-ae2e-45da4b39ead7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:11.250143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16eef0-cdf5-4a06-8629-1eae7a7150d5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:11.271152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976c44c-b760-4e1e-8371-28f4eecad324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:11.271859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976c44c-b760-4e1e-8371-28f4eecad324 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:11.272145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976c44c-b760-4e1e-8371-28f4eecad324 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:11.272426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976c44c-b760-4e1e-8371-28f4eecad324 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:11.273260 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976c44c-b760-4e1e-8371-28f4eecad324 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:11.273600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976c44c-b760-4e1e-8371-28f4eecad324 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:11.275066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976c44c-b760-4e1e-8371-28f4eecad324 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:11.285958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a1c5a-d26b-4b24-a39b-24d508e764d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:11.287601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a1c5a-d26b-4b24-a39b-24d508e764d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:11.288035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a1c5a-d26b-4b24-a39b-24d508e764d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:11.288482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a1c5a-d26b-4b24-a39b-24d508e764d5 None admin] PATH_INFO: `/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/d1ad1ffe4d99420cb568d6154f779838/roles/be88bd87b07244debee4e9545284ff13` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:11.298113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 292/592] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Jul 14 10:54:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:11.298113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976c44c-b760-4e1e-8371-28f4eecad324 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:11.308028 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a1c5a-d26b-4b24-a39b-24d508e764d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a68601f16d1c49548b6e3af0b9819ba3, user_id=d1ad1ffe4d99420cb568d6154f779838, role_id=be88bd87b07244debee4e9545284ff13)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:11.309845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a1c5a-d26b-4b24-a39b-24d508e764d5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:11.328528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 301/593] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:54:11 2022] PUT /identity/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/d1ad1ffe4d99420cb568d6154f779838/roles/be88bd87b07244debee4e9545284ff13 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:54:11.328528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a1c5a-d26b-4b24-a39b-24d508e764d5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:11.328528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:11.407150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:11.452943 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976c44c-b760-4e1e-8371-28f4eecad324 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:11.472689 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a1c5a-d26b-4b24-a39b-24d508e764d5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:11.493879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd37106-7c3d-442f-a929-9a99e0f0c94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:11.494801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd37106-7c3d-442f-a929-9a99e0f0c94f None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:11.495088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd37106-7c3d-442f-a929-9a99e0f0c94f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:11.495371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd37106-7c3d-442f-a929-9a99e0f0c94f None admin] PATH_INFO: `/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/80230b1b73944e8ebb6fec3dbe0d5c4b/roles/be88bd87b07244debee4e9545284ff13` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:11.512065 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd37106-7c3d-442f-a929-9a99e0f0c94f None admin] RBAC: Authorizing `identity:create_grant(project_id=a68601f16d1c49548b6e3af0b9819ba3, user_id=80230b1b73944e8ebb6fec3dbe0d5c4b, role_id=be88bd87b07244debee4e9545284ff13)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:11.512542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d899d-7478-4f33-82f8-9270b901dcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:11.513176 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d899d-7478-4f33-82f8-9270b901dcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:11.513467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d899d-7478-4f33-82f8-9270b901dcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:11.513584 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd37106-7c3d-442f-a929-9a99e0f0c94f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:11.513768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d899d-7478-4f33-82f8-9270b901dcf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:11.514479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d899d-7478-4f33-82f8-9270b901dcf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:11.514791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d899d-7478-4f33-82f8-9270b901dcf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:11.516193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d899d-7478-4f33-82f8-9270b901dcf8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:11.528640 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 302/594] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:11.529609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 293/595] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:54:11 2022] PUT /identity/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/80230b1b73944e8ebb6fec3dbe0d5c4b/roles/be88bd87b07244debee4e9545284ff13 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:54:11.529925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d899d-7478-4f33-82f8-9270b901dcf8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:11.534102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd37106-7c3d-442f-a929-9a99e0f0c94f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:11.610439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:11.652939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d899d-7478-4f33-82f8-9270b901dcf8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:11.664391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd37106-7c3d-442f-a929-9a99e0f0c94f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:11.685842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:11.712320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2e6ef-fb09-460b-b290-48408c19b0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:11.714032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2e6ef-fb09-460b-b290-48408c19b0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:11.714675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2e6ef-fb09-460b-b290-48408c19b0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:11.715336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2e6ef-fb09-460b-b290-48408c19b0d2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:11.716251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2075e-3cf5-4fd9-a69f-560386a92e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:11.717011 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2075e-3cf5-4fd9-a69f-560386a92e37 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:11.717298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2075e-3cf5-4fd9-a69f-560386a92e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:11.717576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2075e-3cf5-4fd9-a69f-560386a92e37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:11.724670 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2e6ef-fb09-460b-b290-48408c19b0d2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:11.726286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2e6ef-fb09-460b-b290-48408c19b0d2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f7e2e6ef-fb09-460b-b290-48408c19b0d2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:11.742706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:11.743722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:11.744661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:11.744661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:11.744661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:11.744661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:11.744661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:11.744661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:11.744661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:11.744661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:11.744661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:54:11.744661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:11.744661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 294/596] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:11.749807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2e6ef-fb09-460b-b290-48408c19b0d2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:11.753500 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2075e-3cf5-4fd9-a69f-560386a92e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:11.755564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2075e-3cf5-4fd9-a69f-560386a92e37 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c7d2075e-3cf5-4fd9-a69f-560386a92e37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:11.785354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:11.786329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:11.787192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:11.787192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:11.787192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:11.787192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:11.789704 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 303/597] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:11.794671 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2075e-3cf5-4fd9-a69f-560386a92e37 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:11.867790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2e6ef-fb09-460b-b290-48408c19b0d2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:11.904113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbbb21-2005-4c77-ba39-b5029f2963be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:11.904814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbbb21-2005-4c77-ba39-b5029f2963be None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:11.905099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbbb21-2005-4c77-ba39-b5029f2963be None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:11.905376 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbbb21-2005-4c77-ba39-b5029f2963be None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:11.906215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbbb21-2005-4c77-ba39-b5029f2963be None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:11.906526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbbb21-2005-4c77-ba39-b5029f2963be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:11.908034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbbb21-2005-4c77-ba39-b5029f2963be None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:11.926943 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 295/598] 10.222.0.75 () {56 vars in 1139 bytes} [Thu Jul 14 10:54:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:11.932006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbbb21-2005-4c77-ba39-b5029f2963be None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:11.935959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2075e-3cf5-4fd9-a69f-560386a92e37 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:11.952279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:11.976609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a5382-6157-4d04-8d25-2fdb0c61a653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:11.977717 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:11.978820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a5382-6157-4d04-8d25-2fdb0c61a653 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:11.979264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a5382-6157-4d04-8d25-2fdb0c61a653 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:11.982300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a5382-6157-4d04-8d25-2fdb0c61a653 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:11.983220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a5382-6157-4d04-8d25-2fdb0c61a653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:11.983757 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a5382-6157-4d04-8d25-2fdb0c61a653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:11.985575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a5382-6157-4d04-8d25-2fdb0c61a653 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:12.000718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 304/599] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:12.006256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a5382-6157-4d04-8d25-2fdb0c61a653 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:12.051560 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbbb21-2005-4c77-ba39-b5029f2963be None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:12.086122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198237bc-a7af-4cdc-a6f4-c9766a264fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:12.087067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198237bc-a7af-4cdc-a6f4-c9766a264fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:12.087483 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198237bc-a7af-4cdc-a6f4-c9766a264fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:12.087880 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198237bc-a7af-4cdc-a6f4-c9766a264fa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:12.098639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198237bc-a7af-4cdc-a6f4-c9766a264fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:12.100260 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198237bc-a7af-4cdc-a6f4-c9766a264fa3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-198237bc-a7af-4cdc-a6f4-c9766a264fa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:12.119692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:12.120801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:12.121905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:12.121905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:12.121905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:12.121905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:12.123705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 296/600] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:12.128701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198237bc-a7af-4cdc-a6f4-c9766a264fa3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:12.216254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a5382-6157-4d04-8d25-2fdb0c61a653 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:12.231783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:12.243974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198237bc-a7af-4cdc-a6f4-c9766a264fa3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:12.280545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb777c0-84bb-4aae-a46c-c2c728730949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:12.281361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb777c0-84bb-4aae-a46c-c2c728730949 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:12.281745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb777c0-84bb-4aae-a46c-c2c728730949 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:12.282121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb777c0-84bb-4aae-a46c-c2c728730949 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:12.282973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb777c0-84bb-4aae-a46c-c2c728730949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:12.283396 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb777c0-84bb-4aae-a46c-c2c728730949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:12.286186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb777c0-84bb-4aae-a46c-c2c728730949 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:12.293012 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfcff8-5ade-44a3-b244-73bdce55df19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:12.293831 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfcff8-5ade-44a3-b244-73bdce55df19 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:12.294121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfcff8-5ade-44a3-b244-73bdce55df19 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:12.294404 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfcff8-5ade-44a3-b244-73bdce55df19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:12.299426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 297/601] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:12.304969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb777c0-84bb-4aae-a46c-c2c728730949 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:12.308373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bfcff8-5ade-44a3-b244-73bdce55df19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:12.309826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bfcff8-5ade-44a3-b244-73bdce55df19 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a4bfcff8-5ade-44a3-b244-73bdce55df19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:12.348001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:12.349027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:12.349896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:12.349896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:12.349896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:12.349896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:12.349896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:12.349896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:12.349896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:12.349896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:12.349896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 305/602] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:12.355293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfcff8-5ade-44a3-b244-73bdce55df19 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:12.399183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:12.447244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb777c0-84bb-4aae-a46c-c2c728730949 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:12.480103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfcff8-5ade-44a3-b244-73bdce55df19 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:12.488571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb36f1c-b607-43c3-a525-a6905190302a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:12.489935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb36f1c-b607-43c3-a525-a6905190302a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:12.490223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb36f1c-b607-43c3-a525-a6905190302a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:12.490501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb36f1c-b607-43c3-a525-a6905190302a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:12.499873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb36f1c-b607-43c3-a525-a6905190302a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:12.501497 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb36f1c-b607-43c3-a525-a6905190302a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-fdb36f1c-b607-43c3-a525-a6905190302a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:12.518961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:12.519893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:12.521391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:12.521391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:12.521391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:12.521391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:12.521391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:12.521391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:12.521391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:12.521391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:12.521391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 298/603] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:12.521865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67824f4-842a-4ad4-bb69-8ccfd4c582ae None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:12.522106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67824f4-842a-4ad4-bb69-8ccfd4c582ae None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:12.522258 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67824f4-842a-4ad4-bb69-8ccfd4c582ae None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:12.522886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e033d-a48d-4bd1-b9a0-584648278d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:12.523818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 299/604] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:12.527805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67824f4-842a-4ad4-bb69-8ccfd4c582ae None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:12.528289 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e033d-a48d-4bd1-b9a0-584648278d84 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:12.528774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e033d-a48d-4bd1-b9a0-584648278d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:12.529331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e033d-a48d-4bd1-b9a0-584648278d84 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:12.530342 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e033d-a48d-4bd1-b9a0-584648278d84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:12.530873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e033d-a48d-4bd1-b9a0-584648278d84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:12.532830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e033d-a48d-4bd1-b9a0-584648278d84 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:12.537493 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:12.549062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 306/605] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:12.550759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d26acb-8c1f-4d4d-8c0b-beed5ef627cc None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:12.551253 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d26acb-8c1f-4d4d-8c0b-beed5ef627cc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:12.552405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d26acb-8c1f-4d4d-8c0b-beed5ef627cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:12.570324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e5d26acb-8c1f-4d4d-8c0b-beed5ef627cc None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:12.572007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e5d26acb-8c1f-4d4d-8c0b-beed5ef627cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:12.646033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67824f4-842a-4ad4-bb69-8ccfd4c582ae None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:12.671411 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 307/606] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:12.671411 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d26acb-8c1f-4d4d-8c0b-beed5ef627cc None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:12.696678 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:12.710136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acc48b-1007-4dfc-a9c2-c349327f6a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:12.711189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc48b-1007-4dfc-a9c2-c349327f6a37 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:12.711713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc48b-1007-4dfc-a9c2-c349327f6a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:12.712260 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc48b-1007-4dfc-a9c2-c349327f6a37 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:12.713267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acc48b-1007-4dfc-a9c2-c349327f6a37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:12.713826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acc48b-1007-4dfc-a9c2-c349327f6a37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:12.716304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acc48b-1007-4dfc-a9c2-c349327f6a37 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:12.735541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 300/607] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:12.736965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acc48b-1007-4dfc-a9c2-c349327f6a37 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:12.793257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d26acb-8c1f-4d4d-8c0b-beed5ef627cc None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:12.809032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:12.842257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e2a29-ec42-4b25-8364-f99d266e4e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:12.843416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e2a29-ec42-4b25-8364-f99d266e4e8b None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:12.843730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e2a29-ec42-4b25-8364-f99d266e4e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:12.844014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e2a29-ec42-4b25-8364-f99d266e4e8b None admin] PATH_INFO: `/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/7e572bf785284a8693cf386ced004524/roles/be88bd87b07244debee4e9545284ff13` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:12.847837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acc48b-1007-4dfc-a9c2-c349327f6a37 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:12.863724 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e2a29-ec42-4b25-8364-f99d266e4e8b None admin] RBAC: Authorizing `identity:create_grant(project_id=a68601f16d1c49548b6e3af0b9819ba3, user_id=7e572bf785284a8693cf386ced004524, role_id=be88bd87b07244debee4e9545284ff13)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:12.866045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e2a29-ec42-4b25-8364-f99d266e4e8b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:12.884428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 308/608] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:54:12 2022] PUT /identity/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/7e572bf785284a8693cf386ced004524/roles/be88bd87b07244debee4e9545284ff13 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:54:12.885231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e2a29-ec42-4b25-8364-f99d266e4e8b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:12.896316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6491a-d190-4708-bd51-f0f867f8fc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:12.897908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6491a-d190-4708-bd51-f0f867f8fc3b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:12.899373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6491a-d190-4708-bd51-f0f867f8fc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:12.899968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6491a-d190-4708-bd51-f0f867f8fc3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:12.925827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6491a-d190-4708-bd51-f0f867f8fc3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f6e6491a-d190-4708-bd51-f0f867f8fc3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:12.940388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:12.941465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:12.942322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:12.942322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:12.942322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:12.942322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:12.942322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:12.942322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:12.942322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:12.942322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:12.942322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:12.942322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:12.942322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:12.942322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:12.946125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 301/609] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:12.964685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6491a-d190-4708-bd51-f0f867f8fc3b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:13.014945 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:13.052772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e2a29-ec42-4b25-8364-f99d266e4e8b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:13.089576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b72b3-2b87-45e4-a9ee-2243775d2a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:13.090498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b72b3-2b87-45e4-a9ee-2243775d2a14 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:13.090783 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b72b3-2b87-45e4-a9ee-2243775d2a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:13.091049 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b72b3-2b87-45e4-a9ee-2243775d2a14 None admin] PATH_INFO: `/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/dadd485b7cac42f4aa079cab91973c35/roles/be88bd87b07244debee4e9545284ff13` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:13.109718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b72b3-2b87-45e4-a9ee-2243775d2a14 None admin] RBAC: Authorizing `identity:create_grant(project_id=a68601f16d1c49548b6e3af0b9819ba3, user_id=dadd485b7cac42f4aa079cab91973c35, role_id=be88bd87b07244debee4e9545284ff13)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:13.111792 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b72b3-2b87-45e4-a9ee-2243775d2a14 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:13.129993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 309/610] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:54:12 2022] PUT /identity/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/dadd485b7cac42f4aa079cab91973c35/roles/be88bd87b07244debee4e9545284ff13 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:54:13.180712 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6491a-d190-4708-bd51-f0f867f8fc3b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:13.217417 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:13.226603 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee795156-71b3-4092-be3d-cf3c084305e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:13.227304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee795156-71b3-4092-be3d-cf3c084305e9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:13.227674 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee795156-71b3-4092-be3d-cf3c084305e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:13.227958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee795156-71b3-4092-be3d-cf3c084305e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:13.228731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee795156-71b3-4092-be3d-cf3c084305e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:13.229043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee795156-71b3-4092-be3d-cf3c084305e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:13.230499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee795156-71b3-4092-be3d-cf3c084305e9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:13.240836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 302/611] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:13.246718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b72b3-2b87-45e4-a9ee-2243775d2a14 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:13.387487 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b72b3-2b87-45e4-a9ee-2243775d2a14 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:13.414362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:13.429532 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0f142-d007-4b9d-b8fa-957720b68cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:13.430334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0f142-d007-4b9d-b8fa-957720b68cac None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:13.430615 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0f142-d007-4b9d-b8fa-957720b68cac None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:13.430889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0f142-d007-4b9d-b8fa-957720b68cac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:13.441507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0f142-d007-4b9d-b8fa-957720b68cac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:13.443555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0f142-d007-4b9d-b8fa-957720b68cac None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b7a0f142-d007-4b9d-b8fa-957720b68cac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:13.461558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:13.462482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:13.463646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:13.463646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:13.463646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:13.463646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:13.463646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 310/612] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:13.468547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee795156-71b3-4092-be3d-cf3c084305e9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:13.616006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee795156-71b3-4092-be3d-cf3c084305e9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:13.660235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fba378-f913-4f64-88ab-09720722a131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:13.660907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fba378-f913-4f64-88ab-09720722a131 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:13.661748 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fba378-f913-4f64-88ab-09720722a131 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:13.662024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fba378-f913-4f64-88ab-09720722a131 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:13.662740 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fba378-f913-4f64-88ab-09720722a131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:13.663045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fba378-f913-4f64-88ab-09720722a131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:13.664482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fba378-f913-4f64-88ab-09720722a131 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:13.683073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 303/613] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:13.688784 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0f142-d007-4b9d-b8fa-957720b68cac None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:13.716626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1265aaf-c4b4-4014-880b-1fc2eac765fb None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:13.716979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1265aaf-c4b4-4014-880b-1fc2eac765fb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:13.717179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1265aaf-c4b4-4014-880b-1fc2eac765fb None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:13.718857 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 304/614] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:54:13.724325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc6ff8-dbe9-4b89-890d-3e5360df3d74 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:13.724611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc6ff8-dbe9-4b89-890d-3e5360df3d74 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:13.724888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc6ff8-dbe9-4b89-890d-3e5360df3d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:13.753207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fbbc6ff8-dbe9-4b89-890d-3e5360df3d74 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:13.754534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fbbc6ff8-dbe9-4b89-890d-3e5360df3d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:13.840033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0f142-d007-4b9d-b8fa-957720b68cac None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:13.855748 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:13.879414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 305/615] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:54:13.904135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d004f9-f4a6-4ca5-ae92-db7ffafd9346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:13.904947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d004f9-f4a6-4ca5-ae92-db7ffafd9346 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:13.905235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d004f9-f4a6-4ca5-ae92-db7ffafd9346 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:13.905521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d004f9-f4a6-4ca5-ae92-db7ffafd9346 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:13.923852 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc6ff8-dbe9-4b89-890d-3e5360df3d74 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:13.932003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d004f9-f4a6-4ca5-ae92-db7ffafd9346 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:13.933800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d004f9-f4a6-4ca5-ae92-db7ffafd9346 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-76d004f9-f4a6-4ca5-ae92-db7ffafd9346 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:13.952873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:13.953932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:13.954911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:13.954911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:13.954911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:13.954911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:13.954911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:13.954911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:13.954911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:13.954911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:13.954911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:54:13.954911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:13.955502 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 311/616] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:54:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:13.964447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d004f9-f4a6-4ca5-ae92-db7ffafd9346 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:14.011294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:14.080153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d004f9-f4a6-4ca5-ae92-db7ffafd9346 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:14.128159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2b111-2d8d-4ab7-ab4d-dff0f1f8c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:14.128845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2b111-2d8d-4ab7-ab4d-dff0f1f8c935 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:14.129131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2b111-2d8d-4ab7-ab4d-dff0f1f8c935 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:14.129405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2b111-2d8d-4ab7-ab4d-dff0f1f8c935 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:14.130793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2b111-2d8d-4ab7-ab4d-dff0f1f8c935 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:14.130902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc6ff8-dbe9-4b89-890d-3e5360df3d74 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:14.131092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2b111-2d8d-4ab7-ab4d-dff0f1f8c935 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:14.132580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2b111-2d8d-4ab7-ab4d-dff0f1f8c935 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:14.169684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5618540-f340-46f5-a4dc-64bca9aa097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:14.170491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5618540-f340-46f5-a4dc-64bca9aa097f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:14.170776 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5618540-f340-46f5-a4dc-64bca9aa097f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:14.170890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 312/617] 10.222.0.75 () {56 vars in 1135 bytes} [Thu Jul 14 10:54:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:14.171068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5618540-f340-46f5-a4dc-64bca9aa097f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:14.176545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2b111-2d8d-4ab7-ab4d-dff0f1f8c935 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:14.182643 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5618540-f340-46f5-a4dc-64bca9aa097f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-c5618540-f340-46f5-a4dc-64bca9aa097f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:14.198024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:14.198958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:14.199819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:14.199819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:14.199819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:14.199819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:14.199819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:14.199819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:14.199819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:14.199819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:14.199819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:14.199819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:14.199819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:14.199819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:14.203723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 306/618] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:14.205972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5618540-f340-46f5-a4dc-64bca9aa097f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:14.263848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:14.316355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:14.328913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2b111-2d8d-4ab7-ab4d-dff0f1f8c935 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:14.344982 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5618540-f340-46f5-a4dc-64bca9aa097f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:14.379063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b93f3-720a-4c06-8ba1-f8f78f8c4684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:14.380454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b93f3-720a-4c06-8ba1-f8f78f8c4684 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:14.380748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b93f3-720a-4c06-8ba1-f8f78f8c4684 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:14.381046 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b93f3-720a-4c06-8ba1-f8f78f8c4684 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:14.386353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905444a8-e36d-4398-a66a-824e10235c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:14.387270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905444a8-e36d-4398-a66a-824e10235c3a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:14.387796 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905444a8-e36d-4398-a66a-824e10235c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:14.388282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905444a8-e36d-4398-a66a-824e10235c3a None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:14.389233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905444a8-e36d-4398-a66a-824e10235c3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:14.389763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905444a8-e36d-4398-a66a-824e10235c3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:14.391330 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905444a8-e36d-4398-a66a-824e10235c3a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:14.392186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b93f3-720a-4c06-8ba1-f8f78f8c4684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:14.393894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b93f3-720a-4c06-8ba1-f8f78f8c4684 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:14.401211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 307/619] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:14.403050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ca0f9-57df-44d5-88ab-74573e73cfba None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:14.403633 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ca0f9-57df-44d5-88ab-74573e73cfba None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:14.404130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ca0f9-57df-44d5-88ab-74573e73cfba None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:14.406164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 308/620] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:14.407139 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ca0f9-57df-44d5-88ab-74573e73cfba None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-013b93f3-720a-4c06-8ba1-f8f78f8c4684 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:14.423423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:14.424389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:14.425270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:14.425270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:14.425270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:14.425270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:14.427472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 313/621] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:14.429030 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516e6ea-2e7b-4313-9eb1-fa5eadf538bc None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:14.429528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516e6ea-2e7b-4313-9eb1-fa5eadf538bc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:14.430815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516e6ea-2e7b-4313-9eb1-fa5eadf538bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:14.450346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b516e6ea-2e7b-4313-9eb1-fa5eadf538bc None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:14.451806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b516e6ea-2e7b-4313-9eb1-fa5eadf538bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:14.535246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 314/622] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:14.536492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516e6ea-2e7b-4313-9eb1-fa5eadf538bc None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:14.557093 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ca0f9-57df-44d5-88ab-74573e73cfba None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:14.562934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:14.598447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42389efc-1d53-493c-b58c-87cacfd87447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:14.599504 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42389efc-1d53-493c-b58c-87cacfd87447 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:14.599631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42389efc-1d53-493c-b58c-87cacfd87447 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:14.599828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42389efc-1d53-493c-b58c-87cacfd87447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:14.610462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42389efc-1d53-493c-b58c-87cacfd87447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:14.612028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42389efc-1d53-493c-b58c-87cacfd87447 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-42389efc-1d53-493c-b58c-87cacfd87447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:14.630939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:14.631853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:14.632707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:14.632707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:14.632707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:14.632707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:14.632707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 309/623] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:14.633428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42389efc-1d53-493c-b58c-87cacfd87447 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:14.646201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:14.675240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516e6ea-2e7b-4313-9eb1-fa5eadf538bc None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:14.714734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252c13b-779b-491c-a074-a8003875395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:14.715434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252c13b-779b-491c-a074-a8003875395a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:14.715733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252c13b-779b-491c-a074-a8003875395a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:14.716050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252c13b-779b-491c-a074-a8003875395a None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:14.716737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252c13b-779b-491c-a074-a8003875395a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:14.717048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252c13b-779b-491c-a074-a8003875395a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:14.718505 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252c13b-779b-491c-a074-a8003875395a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:14.735440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 315/624] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:14.735440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252c13b-779b-491c-a074-a8003875395a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:14.762244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42389efc-1d53-493c-b58c-87cacfd87447 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:14.798161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ce8d6-5aa4-4c07-a7b6-f962f6927bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:14.798833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ce8d6-5aa4-4c07-a7b6-f962f6927bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:14.799118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ce8d6-5aa4-4c07-a7b6-f962f6927bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:14.799411 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ce8d6-5aa4-4c07-a7b6-f962f6927bc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:14.812672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ce8d6-5aa4-4c07-a7b6-f962f6927bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-a10ce8d6-5aa4-4c07-a7b6-f962f6927bc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:14.819995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:14.820934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:14.821770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:14.821770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:14.821770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:14.821770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:14.821770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:14.821770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:14.821770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:14.821770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:14.821770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:14.821770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:14.821770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:14.821770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:14.826185 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 310/625] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:14.826925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ce8d6-5aa4-4c07-a7b6-f962f6927bc3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:14.844011 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:14.869394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252c13b-779b-491c-a074-a8003875395a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:14.905360 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a05623-2252-4614-97aa-881dfe278ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:14.906375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a05623-2252-4614-97aa-881dfe278ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:14.906885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a05623-2252-4614-97aa-881dfe278ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:14.907368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a05623-2252-4614-97aa-881dfe278ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:14.908349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a05623-2252-4614-97aa-881dfe278ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:14.908889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a05623-2252-4614-97aa-881dfe278ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:14.910482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a05623-2252-4614-97aa-881dfe278ad6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:14.913703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:14.927244 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 316/626] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:14.928536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a05623-2252-4614-97aa-881dfe278ad6 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:14.937190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ce8d6-5aa4-4c07-a7b6-f962f6927bc3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:14.972695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a260ee-c38a-4372-a69e-1e868831b8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:14.974040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a260ee-c38a-4372-a69e-1e868831b8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:14.974576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a260ee-c38a-4372-a69e-1e868831b8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:14.974696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a260ee-c38a-4372-a69e-1e868831b8a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:14.981923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a260ee-c38a-4372-a69e-1e868831b8a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:14.983468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a260ee-c38a-4372-a69e-1e868831b8a3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-44a260ee-c38a-4372-a69e-1e868831b8a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:14.998954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:15.000012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:15.001101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:15.001101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:15.001101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:15.001101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:15.001101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:15.001101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:15.001101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:15.001101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:15.001101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 311/627] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:15.001613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a260ee-c38a-4372-a69e-1e868831b8a3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:15.044469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a05623-2252-4614-97aa-881dfe278ad6 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:15.081376 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37308de8-f1dc-42d8-9081-3444d121f428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:15.081606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:15.082431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37308de8-f1dc-42d8-9081-3444d121f428 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:15.083519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37308de8-f1dc-42d8-9081-3444d121f428 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:15.084019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37308de8-f1dc-42d8-9081-3444d121f428 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:15.085184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37308de8-f1dc-42d8-9081-3444d121f428 None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:15.085782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37308de8-f1dc-42d8-9081-3444d121f428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:15.087332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37308de8-f1dc-42d8-9081-3444d121f428 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:15.096705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 317/628] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:15.097753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37308de8-f1dc-42d8-9081-3444d121f428 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:15.116417 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a260ee-c38a-4372-a69e-1e868831b8a3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:15.155117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a06d01-b0eb-4a53-9040-4b3408b2e3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:15.155965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a06d01-b0eb-4a53-9040-4b3408b2e3bd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:15.156252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a06d01-b0eb-4a53-9040-4b3408b2e3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:15.156525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a06d01-b0eb-4a53-9040-4b3408b2e3bd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:15.165910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a06d01-b0eb-4a53-9040-4b3408b2e3bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:15.167443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a06d01-b0eb-4a53-9040-4b3408b2e3bd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:15.169944 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-79a06d01-b0eb-4a53-9040-4b3408b2e3bd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:15.184148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:15.185382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:15.186402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:15.186402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:15.186402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:15.186402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:15.186402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:15.186402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:15.186402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:15.186402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:15.186402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:54:15.186402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:15.186402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 312/629] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:15.187048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a06d01-b0eb-4a53-9040-4b3408b2e3bd None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:15.219499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37308de8-f1dc-42d8-9081-3444d121f428 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:15.255292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2e187-4b7a-4c4a-9fb0-fc561adb906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:15.256250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2e187-4b7a-4c4a-9fb0-fc561adb906f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:15.256720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2e187-4b7a-4c4a-9fb0-fc561adb906f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:15.257152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2e187-4b7a-4c4a-9fb0-fc561adb906f None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:15.258112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2e187-4b7a-4c4a-9fb0-fc561adb906f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:15.258584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2e187-4b7a-4c4a-9fb0-fc561adb906f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:15.260213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2e187-4b7a-4c4a-9fb0-fc561adb906f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:15.278638 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 318/630] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:15.279843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2e187-4b7a-4c4a-9fb0-fc561adb906f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:15.298568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a06d01-b0eb-4a53-9040-4b3408b2e3bd None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:15.351094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c17b0c-a506-43b6-bd9a-d6595ce67a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:15.351842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c17b0c-a506-43b6-bd9a-d6595ce67a43 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:15.352125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c17b0c-a506-43b6-bd9a-d6595ce67a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:15.352401 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c17b0c-a506-43b6-bd9a-d6595ce67a43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:15.354739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:15.365713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c17b0c-a506-43b6-bd9a-d6595ce67a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:15.367169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c17b0c-a506-43b6-bd9a-d6595ce67a43 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-62c17b0c-a506-43b6-bd9a-d6595ce67a43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:15.385458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:15.386605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:15.387530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:15.387530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:15.387530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:15.387530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:15.387530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 313/631] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:15.387530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c17b0c-a506-43b6-bd9a-d6595ce67a43 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:15.400114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2e187-4b7a-4c4a-9fb0-fc561adb906f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:15.419308 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:15.438382 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb8710-1f29-4cc3-a141-dc63782b5bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:15.439937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb8710-1f29-4cc3-a141-dc63782b5bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:15.439937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb8710-1f29-4cc3-a141-dc63782b5bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:15.439937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb8710-1f29-4cc3-a141-dc63782b5bd9 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:15.440716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb8710-1f29-4cc3-a141-dc63782b5bd9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:15.440907 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb8710-1f29-4cc3-a141-dc63782b5bd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:15.443179 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb8710-1f29-4cc3-a141-dc63782b5bd9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:15.468154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 319/632] 10.222.0.75 () {56 vars in 1137 bytes} [Thu Jul 14 10:54:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:15.471630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc904849-7e13-4d2e-ad7f-0702f293baf0 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:15.472115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc904849-7e13-4d2e-ad7f-0702f293baf0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:15.472586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc904849-7e13-4d2e-ad7f-0702f293baf0 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:15.477397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 320/633] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:15.477397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc904849-7e13-4d2e-ad7f-0702f293baf0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:15.523540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c17b0c-a506-43b6-bd9a-d6595ce67a43 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:15.565878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd3bf5-0b62-4fbe-8c55-76a308269b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:15.568226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd3bf5-0b62-4fbe-8c55-76a308269b4a None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:15.568698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd3bf5-0b62-4fbe-8c55-76a308269b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:15.569336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd3bf5-0b62-4fbe-8c55-76a308269b4a None admin] PATH_INFO: `/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/978e71dfbb134f41871114824b03fbeb/roles/be88bd87b07244debee4e9545284ff13` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:15.591594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd3bf5-0b62-4fbe-8c55-76a308269b4a None admin] RBAC: Authorizing `identity:create_grant(project_id=a68601f16d1c49548b6e3af0b9819ba3, user_id=978e71dfbb134f41871114824b03fbeb, role_id=be88bd87b07244debee4e9545284ff13)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:15.593973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd3bf5-0b62-4fbe-8c55-76a308269b4a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:15.608165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc904849-7e13-4d2e-ad7f-0702f293baf0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:15.619800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 314/634] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:54:15 2022] PUT /identity/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/978e71dfbb134f41871114824b03fbeb/roles/be88bd87b07244debee4e9545284ff13 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:54:15.619800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd3bf5-0b62-4fbe-8c55-76a308269b4a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:15.665677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71ec60-933a-4ee8-84c3-e40e6e3704f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:15.666669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71ec60-933a-4ee8-84c3-e40e6e3704f3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:15.666786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:15.667105 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71ec60-933a-4ee8-84c3-e40e6e3704f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:15.667566 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71ec60-933a-4ee8-84c3-e40e6e3704f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:15.669192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71ec60-933a-4ee8-84c3-e40e6e3704f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:15.669662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71ec60-933a-4ee8-84c3-e40e6e3704f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:15.671947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71ec60-933a-4ee8-84c3-e40e6e3704f3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:15.690853 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 321/635] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:15.692607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101ced8-81da-4b5e-90d5-9b89727235dc None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:15.692736 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101ced8-81da-4b5e-90d5-9b89727235dc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:15.693029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101ced8-81da-4b5e-90d5-9b89727235dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:15.715219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5101ced8-81da-4b5e-90d5-9b89727235dc None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:15.717021 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5101ced8-81da-4b5e-90d5-9b89727235dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:15.748009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd3bf5-0b62-4fbe-8c55-76a308269b4a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:15.750799 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:15.796177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd932fa-c768-4d8f-9846-84ce5c44a1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:15.797000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd932fa-c768-4d8f-9846-84ce5c44a1dc None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:15.797577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd932fa-c768-4d8f-9846-84ce5c44a1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:15.797879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd932fa-c768-4d8f-9846-84ce5c44a1dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:15.805203 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 322/636] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:15.808393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ee001-260b-4655-94fd-ffedb5e30897 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:15.808393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ee001-260b-4655-94fd-ffedb5e30897 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:15.808393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ee001-260b-4655-94fd-ffedb5e30897 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:15.808664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 323/637] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:15.809371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ee001-260b-4655-94fd-ffedb5e30897 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:15.896874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd932fa-c768-4d8f-9846-84ce5c44a1dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:15.898971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd932fa-c768-4d8f-9846-84ce5c44a1dc None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:15.940156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:15.941540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ee001-260b-4655-94fd-ffedb5e30897 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:15.977890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf71c2-6e3c-456b-8885-1a939b93f5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:15.978694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf71c2-6e3c-456b-8885-1a939b93f5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:15.978981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf71c2-6e3c-456b-8885-1a939b93f5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:15.979260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf71c2-6e3c-456b-8885-1a939b93f5f0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:15.986756 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf71c2-6e3c-456b-8885-1a939b93f5f0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:15.988245 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf71c2-6e3c-456b-8885-1a939b93f5f0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-efbf71c2-6e3c-456b-8885-1a939b93f5f0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:16.002789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:16.003819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:16.004791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:16.004791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:16.004791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:16.004791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:16.004791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:16.004791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:16.004791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:16.004791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:16.004791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:54:16.004791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:16.004791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 324/638] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:16.005384 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7603f3-1c11-452f-a3d9-c7442210718d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:16.005539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7603f3-1c11-452f-a3d9-c7442210718d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:16.006120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7603f3-1c11-452f-a3d9-c7442210718d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:16.018576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 315/639] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:54:16.019776 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd932fa-c768-4d8f-9846-84ce5c44a1dc None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:16.030347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-af7603f3-1c11-452f-a3d9-c7442210718d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:16.031739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-af7603f3-1c11-452f-a3d9-c7442210718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:16.056997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:16.120564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 325/640] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:16.121450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7603f3-1c11-452f-a3d9-c7442210718d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:16.157832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd932fa-c768-4d8f-9846-84ce5c44a1dc None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:16.201115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b3a04-6e36-42d2-84b0-7c365b291924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:16.201911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b3a04-6e36-42d2-84b0-7c365b291924 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:16.202191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b3a04-6e36-42d2-84b0-7c365b291924 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:16.202463 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b3a04-6e36-42d2-84b0-7c365b291924 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:16.205299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:16.217259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b3a04-6e36-42d2-84b0-7c365b291924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-4c4b3a04-6e36-42d2-84b0-7c365b291924 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:16.225120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:16.226075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:16.226900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:16.226900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:16.226900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:16.226900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:16.226900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:16.226900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:16.226900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:16.226900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:16.226900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:16.226900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:16.226900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:16.226900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:16.231258 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 316/641] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:16.232482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b3a04-6e36-42d2-84b0-7c365b291924 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:16.260946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7603f3-1c11-452f-a3d9-c7442210718d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:16.305218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb4b9a-9e95-4504-9c84-d4b5ee1ac3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:16.306496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb4b9a-9e95-4504-9c84-d4b5ee1ac3af None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:16.307042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb4b9a-9e95-4504-9c84-d4b5ee1ac3af None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:16.307467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb4b9a-9e95-4504-9c84-d4b5ee1ac3af None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:16.309073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb4b9a-9e95-4504-9c84-d4b5ee1ac3af None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:16.309692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb4b9a-9e95-4504-9c84-d4b5ee1ac3af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:16.311958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb4b9a-9e95-4504-9c84-d4b5ee1ac3af None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:16.341296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 326/642] 10.222.0.75 () {56 vars in 1137 bytes} [Thu Jul 14 10:54:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:16.341296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb4b9a-9e95-4504-9c84-d4b5ee1ac3af None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:16.356675 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:16.357602 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b3a04-6e36-42d2-84b0-7c365b291924 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:16.399707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e93312-388c-4e7a-bfd2-5720dd305ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:16.400907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e93312-388c-4e7a-bfd2-5720dd305ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:16.401329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e93312-388c-4e7a-bfd2-5720dd305ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:16.401786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e93312-388c-4e7a-bfd2-5720dd305ca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:16.415293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e93312-388c-4e7a-bfd2-5720dd305ca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:16.416738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e93312-388c-4e7a-bfd2-5720dd305ca5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-99e93312-388c-4e7a-bfd2-5720dd305ca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:16.434356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:16.435574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:16.436811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:16.436811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:16.436811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:16.436811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:16.440140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 317/643] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:16.440927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e93312-388c-4e7a-bfd2-5720dd305ca5 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:16.469132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb4b9a-9e95-4504-9c84-d4b5ee1ac3af None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:16.503360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:16.524219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe5628-5352-42f4-a18d-0b16ece89eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:16.526068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe5628-5352-42f4-a18d-0b16ece89eaa None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:16.526850 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe5628-5352-42f4-a18d-0b16ece89eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:16.527322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe5628-5352-42f4-a18d-0b16ece89eaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:16.538920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe5628-5352-42f4-a18d-0b16ece89eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-50fe5628-5352-42f4-a18d-0b16ece89eaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:16.546890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:16.547930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:16.548964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 327/644] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:16.550357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe5628-5352-42f4-a18d-0b16ece89eaa None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:16.576579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e93312-388c-4e7a-bfd2-5720dd305ca5 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:16.614900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404817cd-3dde-43e8-a0fe-276e5434d407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:16.615613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404817cd-3dde-43e8-a0fe-276e5434d407 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:16.615897 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404817cd-3dde-43e8-a0fe-276e5434d407 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:16.616171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404817cd-3dde-43e8-a0fe-276e5434d407 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:16.616926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404817cd-3dde-43e8-a0fe-276e5434d407 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:16.617231 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404817cd-3dde-43e8-a0fe-276e5434d407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:16.618618 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404817cd-3dde-43e8-a0fe-276e5434d407 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:16.632324 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 318/645] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:16.632324 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404817cd-3dde-43e8-a0fe-276e5434d407 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:16.653431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:16.683824 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe5628-5352-42f4-a18d-0b16ece89eaa None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:16.724294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1ce75-0839-4cde-b9a5-f776f2a966ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:16.725036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1ce75-0839-4cde-b9a5-f776f2a966ee None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:16.725493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1ce75-0839-4cde-b9a5-f776f2a966ee None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:16.725716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1ce75-0839-4cde-b9a5-f776f2a966ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:16.748975 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404817cd-3dde-43e8-a0fe-276e5434d407 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:16.762236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:16.790615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d4b76-7035-4b27-a019-94a89280dd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:16.791288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d4b76-7035-4b27-a019-94a89280dd99 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:16.791616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d4b76-7035-4b27-a019-94a89280dd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:16.791900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d4b76-7035-4b27-a019-94a89280dd99 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:16.792629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d4b76-7035-4b27-a019-94a89280dd99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:16.792939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d4b76-7035-4b27-a019-94a89280dd99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:16.794338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d4b76-7035-4b27-a019-94a89280dd99 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:16.810886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 319/646] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:16.811806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d4b76-7035-4b27-a019-94a89280dd99 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:16.816697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1ce75-0839-4cde-b9a5-f776f2a966ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:16.818784 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1ce75-0839-4cde-b9a5-f776f2a966ee None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:16.906014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:16.938021 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d4b76-7035-4b27-a019-94a89280dd99 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:16.953732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 328/647] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:54:16.953732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1ce75-0839-4cde-b9a5-f776f2a966ee None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:16.985889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5437486-39e6-4791-8993-932b9bf7e708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:16.986679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5437486-39e6-4791-8993-932b9bf7e708 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:16.986820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5437486-39e6-4791-8993-932b9bf7e708 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:16.987140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5437486-39e6-4791-8993-932b9bf7e708 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:16.987853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5437486-39e6-4791-8993-932b9bf7e708 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:16.988428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5437486-39e6-4791-8993-932b9bf7e708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:16.990207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5437486-39e6-4791-8993-932b9bf7e708 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:16.998911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 320/648] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:16.999639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5437486-39e6-4791-8993-932b9bf7e708 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:17.033920 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:17.074237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1ce75-0839-4cde-b9a5-f776f2a966ee None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:17.108183 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5437486-39e6-4791-8993-932b9bf7e708 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:17.115189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b601b-61db-4b5f-9d19-20a0c20bc498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:17.116014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b601b-61db-4b5f-9d19-20a0c20bc498 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:17.116292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b601b-61db-4b5f-9d19-20a0c20bc498 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:17.116571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b601b-61db-4b5f-9d19-20a0c20bc498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:17.128997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b601b-61db-4b5f-9d19-20a0c20bc498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:17.130501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b601b-61db-4b5f-9d19-20a0c20bc498 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:17.147068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871003b2-7b8f-478e-acca-6f70dbae837f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:17.147906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871003b2-7b8f-478e-acca-6f70dbae837f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:17.148193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871003b2-7b8f-478e-acca-6f70dbae837f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:17.148471 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871003b2-7b8f-478e-acca-6f70dbae837f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b68b601b-61db-4b5f-9d19-20a0c20bc498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:17.150862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:17.152740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:17.155099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:17.155099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:17.155099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:17.155099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:17.155099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 329/649] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:17.155099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b601b-61db-4b5f-9d19-20a0c20bc498 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:17.161852 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871003b2-7b8f-478e-acca-6f70dbae837f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:17.163334 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871003b2-7b8f-478e-acca-6f70dbae837f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-871003b2-7b8f-478e-acca-6f70dbae837f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:17.179165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:17.180160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:17.186533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:17.186533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:17.186533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:17.186533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:17.186533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:17.186533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:17.186533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:17.186533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:17.186533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 321/650] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:17.186533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871003b2-7b8f-478e-acca-6f70dbae837f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:17.194971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:17.296608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b601b-61db-4b5f-9d19-20a0c20bc498 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:17.299368 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:17.310496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871003b2-7b8f-478e-acca-6f70dbae837f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:17.346653 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d79ce-3662-43b4-b4b5-05ef447d05ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:17.347448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d79ce-3662-43b4-b4b5-05ef447d05ee None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:17.347733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d79ce-3662-43b4-b4b5-05ef447d05ee None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:17.347867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545e8b0-18d0-4b85-a9ad-a692da91fbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:17.348056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d79ce-3662-43b4-b4b5-05ef447d05ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:17.349210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545e8b0-18d0-4b85-a9ad-a692da91fbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:17.349495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545e8b0-18d0-4b85-a9ad-a692da91fbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:17.349774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545e8b0-18d0-4b85-a9ad-a692da91fbe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:17.360027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d79ce-3662-43b4-b4b5-05ef447d05ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:17.361492 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d79ce-3662-43b4-b4b5-05ef447d05ee None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:17.361808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545e8b0-18d0-4b85-a9ad-a692da91fbe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:17.363271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545e8b0-18d0-4b85-a9ad-a692da91fbe4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b545e8b0-18d0-4b85-a9ad-a692da91fbe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:17.381211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:17.382187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:17.388076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:17.388076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:17.388076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:17.388076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f83d79ce-3662-43b4-b4b5-05ef447d05ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:17.388304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:17.389132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:17.390038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:17.390038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:17.390038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:17.390038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:17.390333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 330/651] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:17.390414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 322/652] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:17.390414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d79ce-3662-43b4-b4b5-05ef447d05ee None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:17.390544 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545e8b0-18d0-4b85-a9ad-a692da91fbe4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:17.503935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:17.511860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d79ce-3662-43b4-b4b5-05ef447d05ee None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:17.534398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545e8b0-18d0-4b85-a9ad-a692da91fbe4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:17.553613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4858e-9760-4808-a093-2feb4f70e436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:17.554590 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4858e-9760-4808-a093-2feb4f70e436 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:17.555096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4858e-9760-4808-a093-2feb4f70e436 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:17.555374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4858e-9760-4808-a093-2feb4f70e436 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:17.556202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4858e-9760-4808-a093-2feb4f70e436 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:17.556511 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4858e-9760-4808-a093-2feb4f70e436 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:17.558247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4858e-9760-4808-a093-2feb4f70e436 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:17.571752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 323/653] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:17.572616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4858e-9760-4808-a093-2feb4f70e436 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:17.576397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:17.586275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150cbca-4450-4aa9-8b50-f561218c983a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:17.586954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150cbca-4450-4aa9-8b50-f561218c983a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:17.587233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150cbca-4450-4aa9-8b50-f561218c983a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:17.587540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150cbca-4450-4aa9-8b50-f561218c983a None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:17.588274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150cbca-4450-4aa9-8b50-f561218c983a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:17.588584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150cbca-4450-4aa9-8b50-f561218c983a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:17.589985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150cbca-4450-4aa9-8b50-f561218c983a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:17.605430 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 331/654] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:17.610137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150cbca-4450-4aa9-8b50-f561218c983a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:17.713270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4858e-9760-4808-a093-2feb4f70e436 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:17.717173 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150cbca-4450-4aa9-8b50-f561218c983a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:17.750720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6213f3-b17a-4d2c-a05d-c07a2cceaffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:17.751401 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6213f3-b17a-4d2c-a05d-c07a2cceaffe None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:17.751703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6213f3-b17a-4d2c-a05d-c07a2cceaffe None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:17.751981 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6213f3-b17a-4d2c-a05d-c07a2cceaffe None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:17.752708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6213f3-b17a-4d2c-a05d-c07a2cceaffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:17.753014 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6213f3-b17a-4d2c-a05d-c07a2cceaffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:17.754466 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6213f3-b17a-4d2c-a05d-c07a2cceaffe None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:17.756524 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b33bef-ea6d-4a3f-bfd7-1995ece69c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:17.757203 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b33bef-ea6d-4a3f-bfd7-1995ece69c35 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:17.757489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b33bef-ea6d-4a3f-bfd7-1995ece69c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:17.757766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b33bef-ea6d-4a3f-bfd7-1995ece69c35 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:17.758493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b33bef-ea6d-4a3f-bfd7-1995ece69c35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:17.758800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b33bef-ea6d-4a3f-bfd7-1995ece69c35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:17.760260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b33bef-ea6d-4a3f-bfd7-1995ece69c35 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:17.766544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 324/655] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:17.767374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6213f3-b17a-4d2c-a05d-c07a2cceaffe None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:17.773541 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:17.779484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 332/656] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:17.780343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b33bef-ea6d-4a3f-bfd7-1995ece69c35 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:17.835251 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:17.877070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6213f3-b17a-4d2c-a05d-c07a2cceaffe None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:17.886343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b33bef-ea6d-4a3f-bfd7-1995ece69c35 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:17.911860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e2e5d-f4e1-4754-a326-1bd1933fd253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:17.912518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e2e5d-f4e1-4754-a326-1bd1933fd253 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:17.912804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e2e5d-f4e1-4754-a326-1bd1933fd253 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:17.913083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e2e5d-f4e1-4754-a326-1bd1933fd253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:17.913911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e2e5d-f4e1-4754-a326-1bd1933fd253 None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=d1ad1ffe4d99420cb568d6154f779838) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:17.914288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e2e5d-f4e1-4754-a326-1bd1933fd253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:17.915779 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e2e5d-f4e1-4754-a326-1bd1933fd253 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:17.921837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 325/657] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:17 2022] GET /identity/v3/role_assignments?user.id=d1ad1ffe4d99420cb568d6154f779838&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:17.924888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f5e10-e96a-4e19-84a0-9a29092602bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:17.926096 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f5e10-e96a-4e19-84a0-9a29092602bd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:17.926554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166852a3-72cb-479c-bcd2-76e8eb48dd6e None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:17.926843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166852a3-72cb-479c-bcd2-76e8eb48dd6e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:17.927335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166852a3-72cb-479c-bcd2-76e8eb48dd6e None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:17.927557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f5e10-e96a-4e19-84a0-9a29092602bd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:17.927812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f5e10-e96a-4e19-84a0-9a29092602bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:17.928718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 326/658] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:17.929394 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166852a3-72cb-479c-bcd2-76e8eb48dd6e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:17.940961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f5e10-e96a-4e19-84a0-9a29092602bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:17.943104 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f5e10-e96a-4e19-84a0-9a29092602bd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-2e7f5e10-e96a-4e19-84a0-9a29092602bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:17.961302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:17.963054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:17.964414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:17.964414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:17.964414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:17.964414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:17.964414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:17.964414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:17.964414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:17.964414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:17.964414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:54:17.964414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:17.964414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 333/659] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Jul 14 10:54:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:17.964414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f5e10-e96a-4e19-84a0-9a29092602bd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:18.045053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:18.079963 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166852a3-72cb-479c-bcd2-76e8eb48dd6e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:18.112282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f5e10-e96a-4e19-84a0-9a29092602bd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:18.121148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:18.135971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce9176-1758-43d4-a760-560690195473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:18.138008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce9176-1758-43d4-a760-560690195473 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:18.138008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce9176-1758-43d4-a760-560690195473 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:18.138008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce9176-1758-43d4-a760-560690195473 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:18.146337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce9176-1758-43d4-a760-560690195473 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:18.147850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce9176-1758-43d4-a760-560690195473 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-c5ce9176-1758-43d4-a760-560690195473 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:18.164057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:18.165080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:18.165995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:18.165995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:18.165995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:18.165995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:18.165995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:18.165995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:18.165995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:18.165995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:18.166421 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c7b2f-0c9d-415b-aa0b-d445e031464a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:18.166511 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 327/660] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:18.166588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c7b2f-0c9d-415b-aa0b-d445e031464a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:18.166588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c7b2f-0c9d-415b-aa0b-d445e031464a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:18.167231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c7b2f-0c9d-415b-aa0b-d445e031464a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:18.170436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99014271-0259-4ca3-a7bc-3543a61f01ca None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:18.170436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99014271-0259-4ca3-a7bc-3543a61f01ca None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:18.170436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99014271-0259-4ca3-a7bc-3543a61f01ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:18.177514 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c7b2f-0c9d-415b-aa0b-d445e031464a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:18.178980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c7b2f-0c9d-415b-aa0b-d445e031464a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:18.187693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-99014271-0259-4ca3-a7bc-3543a61f01ca None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:18.188908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-99014271-0259-4ca3-a7bc-3543a61f01ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-2a5c7b2f-0c9d-415b-aa0b-d445e031464a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:18.196576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:18.197596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:18.198540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:18.198540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:18.198540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:18.198540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:18.198540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:18.198540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:18.198540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:18.198540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:18.198540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:54:18.198540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:18.198540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 334/661] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:18.199135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c7b2f-0c9d-415b-aa0b-d445e031464a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:18.271719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 328/662] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:18.271719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99014271-0259-4ca3-a7bc-3543a61f01ca None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:18.340491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c7b2f-0c9d-415b-aa0b-d445e031464a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:18.358697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:18.388036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990660a5-de6c-48f5-a517-06fe3772749f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:18.388657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990660a5-de6c-48f5-a517-06fe3772749f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:18.388939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990660a5-de6c-48f5-a517-06fe3772749f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:18.389226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990660a5-de6c-48f5-a517-06fe3772749f None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:18.390076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990660a5-de6c-48f5-a517-06fe3772749f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:18.390388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990660a5-de6c-48f5-a517-06fe3772749f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:18.391862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990660a5-de6c-48f5-a517-06fe3772749f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:18.392537 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99014271-0259-4ca3-a7bc-3543a61f01ca None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:18.421445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 335/663] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Jul 14 10:54:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:18.421445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990660a5-de6c-48f5-a517-06fe3772749f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:18.422895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:18.451697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9e34b-5a4b-4ff4-a090-e057526b72d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:18.452424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9e34b-5a4b-4ff4-a090-e057526b72d9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:18.452714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9e34b-5a4b-4ff4-a090-e057526b72d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:18.453003 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9e34b-5a4b-4ff4-a090-e057526b72d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:18.453804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9e34b-5a4b-4ff4-a090-e057526b72d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:18.454117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9e34b-5a4b-4ff4-a090-e057526b72d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:18.455605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9e34b-5a4b-4ff4-a090-e057526b72d9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:18.469497 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 329/664] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:18.470486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9e34b-5a4b-4ff4-a090-e057526b72d9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:18.550834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990660a5-de6c-48f5-a517-06fe3772749f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:18.594406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b642ef-b369-413c-ab9e-482c2ef012f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:18.594997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b642ef-b369-413c-ab9e-482c2ef012f2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:18.595280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b642ef-b369-413c-ab9e-482c2ef012f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:18.595595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b642ef-b369-413c-ab9e-482c2ef012f2 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:18.596431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b642ef-b369-413c-ab9e-482c2ef012f2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:18.596741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b642ef-b369-413c-ab9e-482c2ef012f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:18.598194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b642ef-b369-413c-ab9e-482c2ef012f2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:18.603321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9e34b-5a4b-4ff4-a090-e057526b72d9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:18.620749 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 336/665] 10.222.0.75 () {56 vars in 1139 bytes} [Thu Jul 14 10:54:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:18.621762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b642ef-b369-413c-ab9e-482c2ef012f2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:18.647016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f65b42-98ae-4c19-b586-36d7b7699566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:18.647951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f65b42-98ae-4c19-b586-36d7b7699566 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:18.648402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f65b42-98ae-4c19-b586-36d7b7699566 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:18.648644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f65b42-98ae-4c19-b586-36d7b7699566 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:18.660236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f65b42-98ae-4c19-b586-36d7b7699566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:18.660694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-15f65b42-98ae-4c19-b586-36d7b7699566 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:18.670820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:18.672047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 330/666] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:18.678661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f65b42-98ae-4c19-b586-36d7b7699566 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:18.734265 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:18.753643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b642ef-b369-413c-ab9e-482c2ef012f2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:18.800258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fb3b3-a81e-47d1-8abb-306461f0f2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:18.801014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fb3b3-a81e-47d1-8abb-306461f0f2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:18.801371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fb3b3-a81e-47d1-8abb-306461f0f2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:18.802275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fb3b3-a81e-47d1-8abb-306461f0f2f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:18.819267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f65b42-98ae-4c19-b586-36d7b7699566 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:18.861218 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d58ea-06ce-4ff9-bb63-b9afa57549bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:18.862112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d58ea-06ce-4ff9-bb63-b9afa57549bc None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:18.862539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d58ea-06ce-4ff9-bb63-b9afa57549bc None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:18.862966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d58ea-06ce-4ff9-bb63-b9afa57549bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:18.863830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d58ea-06ce-4ff9-bb63-b9afa57549bc None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=80230b1b73944e8ebb6fec3dbe0d5c4b) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:18.864173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d58ea-06ce-4ff9-bb63-b9afa57549bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:18.865638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d58ea-06ce-4ff9-bb63-b9afa57549bc None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:18.879672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 331/667] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:18 2022] GET /identity/v3/role_assignments?user.id=80230b1b73944e8ebb6fec3dbe0d5c4b&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:18.879672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d58ea-06ce-4ff9-bb63-b9afa57549bc None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:18.890244 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4fb3b3-a81e-47d1-8abb-306461f0f2f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:18.891810 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4fb3b3-a81e-47d1-8abb-306461f0f2f8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:18.940484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:19.033359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d58ea-06ce-4ff9-bb63-b9afa57549bc None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:19.035314 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:19.050181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 337/668] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:54:19.052347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fb3b3-a81e-47d1-8abb-306461f0f2f8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:19.088082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906b738-6826-47bb-9d33-88956c138bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:19.088611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b738-6826-47bb-9d33-88956c138bcc None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:19.088906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b738-6826-47bb-9d33-88956c138bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:19.089185 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b738-6826-47bb-9d33-88956c138bcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:19.185766 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906b738-6826-47bb-9d33-88956c138bcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:19.187514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906b738-6826-47bb-9d33-88956c138bcc None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:19.208669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fb3b3-a81e-47d1-8abb-306461f0f2f8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:19.257845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880e521-70a3-43aa-919a-e964961807a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:19.258537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880e521-70a3-43aa-919a-e964961807a0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:19.258823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880e521-70a3-43aa-919a-e964961807a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:19.259106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880e521-70a3-43aa-919a-e964961807a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:19.260132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880e521-70a3-43aa-919a-e964961807a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:19.260441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880e521-70a3-43aa-919a-e964961807a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:19.262472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880e521-70a3-43aa-919a-e964961807a0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:19.264368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:19.279467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 338/669] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:19.280643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880e521-70a3-43aa-919a-e964961807a0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:19.335553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 332/670] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:54:19.336891 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906b738-6826-47bb-9d33-88956c138bcc None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:19.340420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:19.420015 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880e521-70a3-43aa-919a-e964961807a0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:19.467755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906b738-6826-47bb-9d33-88956c138bcc None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:19.472754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43d596-0a1d-41cf-9ba1-fab56825a68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:19.473609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43d596-0a1d-41cf-9ba1-fab56825a68e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:19.473900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43d596-0a1d-41cf-9ba1-fab56825a68e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:19.474180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43d596-0a1d-41cf-9ba1-fab56825a68e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:19.491145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43d596-0a1d-41cf-9ba1-fab56825a68e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:19.492696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43d596-0a1d-41cf-9ba1-fab56825a68e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:19.514599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef46dc-91e6-4b1d-83e2-d9222e9e8060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-dc43d596-0a1d-41cf-9ba1-fab56825a68e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:19.515312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:19.516185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:19.517073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:19.517073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:19.517073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:19.517073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:19.517286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef46dc-91e6-4b1d-83e2-d9222e9e8060 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:19.517286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef46dc-91e6-4b1d-83e2-d9222e9e8060 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:19.517286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef46dc-91e6-4b1d-83e2-d9222e9e8060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:19.519156 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 339/671] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:19.520908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43d596-0a1d-41cf-9ba1-fab56825a68e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:19.530615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef46dc-91e6-4b1d-83e2-d9222e9e8060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:19.533126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef46dc-91e6-4b1d-83e2-d9222e9e8060 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-1bef46dc-91e6-4b1d-83e2-d9222e9e8060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:19.555612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:19.556581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:19.557452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:19.557452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:19.557452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:19.557452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:19.559457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 333/672] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:19.561094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef46dc-91e6-4b1d-83e2-d9222e9e8060 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:19.588315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:19.625527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:19.649420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43d596-0a1d-41cf-9ba1-fab56825a68e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:19.682117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef46dc-91e6-4b1d-83e2-d9222e9e8060 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:19.688639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a6870-ceb2-47bd-818f-3c636756c05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:19.689460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a6870-ceb2-47bd-818f-3c636756c05a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:19.689749 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a6870-ceb2-47bd-818f-3c636756c05a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:19.690033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a6870-ceb2-47bd-818f-3c636756c05a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:19.703039 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a6870-ceb2-47bd-818f-3c636756c05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:19.704635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a6870-ceb2-47bd-818f-3c636756c05a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:19.721872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86da42-8bf5-45e6-b82e-1fc1d2ed25c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:19.722880 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86da42-8bf5-45e6-b82e-1fc1d2ed25c4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:19.723423 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86da42-8bf5-45e6-b82e-1fc1d2ed25c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:19.723952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86da42-8bf5-45e6-b82e-1fc1d2ed25c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:19.724889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86da42-8bf5-45e6-b82e-1fc1d2ed25c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:19.725396 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86da42-8bf5-45e6-b82e-1fc1d2ed25c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:19.727007 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86da42-8bf5-45e6-b82e-1fc1d2ed25c4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-dd6a6870-ceb2-47bd-818f-3c636756c05a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:19.731539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:19.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:19.733332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:19.733332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:19.733332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:19.733332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:19.733558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 340/673] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:19.734586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a6870-ceb2-47bd-818f-3c636756c05a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:19.744345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 334/674] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:19.745538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86da42-8bf5-45e6-b82e-1fc1d2ed25c4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:19.846886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:19.851126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a6870-ceb2-47bd-818f-3c636756c05a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:19.857023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86da42-8bf5-45e6-b82e-1fc1d2ed25c4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:19.875841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:19.892546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ab8e8-0d8a-4c72-892d-fae5c888a30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:19.893483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ab8e8-0d8a-4c72-892d-fae5c888a30a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:19.893642 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ab8e8-0d8a-4c72-892d-fae5c888a30a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:19.893892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ab8e8-0d8a-4c72-892d-fae5c888a30a None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:19.894617 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ab8e8-0d8a-4c72-892d-fae5c888a30a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:19.894940 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ab8e8-0d8a-4c72-892d-fae5c888a30a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:19.896397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ab8e8-0d8a-4c72-892d-fae5c888a30a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:19.904627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3702789-1f21-4b51-9286-d4a4affda7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:19.905686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3702789-1f21-4b51-9286-d4a4affda7cd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:19.906170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3702789-1f21-4b51-9286-d4a4affda7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:19.906658 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3702789-1f21-4b51-9286-d4a4affda7cd None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:19.907647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3702789-1f21-4b51-9286-d4a4affda7cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:19.908153 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3702789-1f21-4b51-9286-d4a4affda7cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:19.909823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3702789-1f21-4b51-9286-d4a4affda7cd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:19.914656 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 341/675] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:19.915649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ab8e8-0d8a-4c72-892d-fae5c888a30a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:19.923320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 335/676] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:19.924542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3702789-1f21-4b51-9286-d4a4affda7cd None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:20.035295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ab8e8-0d8a-4c72-892d-fae5c888a30a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:20.045864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3702789-1f21-4b51-9286-d4a4affda7cd None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:20.077801 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04699efc-6f92-4866-9843-4097550e19be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:20.078611 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04699efc-6f92-4866-9843-4097550e19be None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:20.078900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04699efc-6f92-4866-9843-4097550e19be None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:20.079179 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04699efc-6f92-4866-9843-4097550e19be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:20.085353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fe155-049e-4d85-be38-2ec2367454af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:20.086474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fe155-049e-4d85-be38-2ec2367454af None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:20.087000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fe155-049e-4d85-be38-2ec2367454af None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:20.087524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fe155-049e-4d85-be38-2ec2367454af None admin] PATH_INFO: `/v3/users/swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:20.087768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04699efc-6f92-4866-9843-4097550e19be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:20.090101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04699efc-6f92-4866-9843-4097550e19be None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:20.097010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fe155-049e-4d85-be38-2ec2367454af None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:20.100783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fe155-049e-4d85-be38-2ec2367454af None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-04699efc-6f92-4866-9843-4097550e19be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:20.106820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:20.107836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:20.108753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:20.108753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:20.108753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:20.108753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:20.108753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:20.108753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:20.108753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:20.108753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:20.108753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 342/677] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:20.109722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04699efc-6f92-4866-9843-4097550e19be None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-2f7fe155-049e-4d85-be38-2ec2367454af None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:20.120990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:20.121995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:20.122923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:20.122923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:20.122923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:20.122923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:20.122923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:20.122923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:20.122923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:20.122923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:20.122923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:54:20.122923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:20.122923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 336/678] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:54:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:20.124365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fe155-049e-4d85-be38-2ec2367454af None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:20.133983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:20.170111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:20.244763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04699efc-6f92-4866-9843-4097550e19be None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:20.248408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fe155-049e-4d85-be38-2ec2367454af None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:20.283364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e34ce4-4d56-4c16-8f21-d554ad6c3144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:20.284193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e34ce4-4d56-4c16-8f21-d554ad6c3144 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:20.284472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e34ce4-4d56-4c16-8f21-d554ad6c3144 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:20.284750 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e34ce4-4d56-4c16-8f21-d554ad6c3144 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:20.285168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38667108-18b7-4422-bcb1-1f0984574d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:20.286440 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38667108-18b7-4422-bcb1-1f0984574d90 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:20.286664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38667108-18b7-4422-bcb1-1f0984574d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:20.286933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38667108-18b7-4422-bcb1-1f0984574d90 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:20.287762 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38667108-18b7-4422-bcb1-1f0984574d90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:20.288072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38667108-18b7-4422-bcb1-1f0984574d90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:20.289531 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38667108-18b7-4422-bcb1-1f0984574d90 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:20.293445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e34ce4-4d56-4c16-8f21-d554ad6c3144 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:20.294909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e34ce4-4d56-4c16-8f21-d554ad6c3144 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:20.302534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 337/679] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:20.303691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d64f73-0195-468e-85dd-9b15aa44d508 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:20.303984 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d64f73-0195-468e-85dd-9b15aa44d508 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:20.304268 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d64f73-0195-468e-85dd-9b15aa44d508 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:20.310364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 338/680] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:20.310364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d64f73-0195-468e-85dd-9b15aa44d508 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-86e34ce4-4d56-4c16-8f21-d554ad6c3144 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:20.318701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:20.319657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:20.320537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:20.320537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:20.320537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:20.320537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:20.320537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:20.320537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:20.320537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:20.320537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:20.323741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 343/681] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:20.325322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e34ce4-4d56-4c16-8f21-d554ad6c3144 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:20.421068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:20.434426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:20.442860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e34ce4-4d56-4c16-8f21-d554ad6c3144 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:20.448680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d64f73-0195-468e-85dd-9b15aa44d508 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:20.487798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bce00-b59f-4734-a692-46800b322c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:20.488477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bce00-b59f-4734-a692-46800b322c48 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:20.488803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bce00-b59f-4734-a692-46800b322c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:20.489075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bce00-b59f-4734-a692-46800b322c48 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:20.489904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bce00-b59f-4734-a692-46800b322c48 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:20.490217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bce00-b59f-4734-a692-46800b322c48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:20.490471 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb8a1e-f3d4-4581-926d-551107fa768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:20.491133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb8a1e-f3d4-4581-926d-551107fa768c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:20.491480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb8a1e-f3d4-4581-926d-551107fa768c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:20.491729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bce00-b59f-4734-a692-46800b322c48 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:20.491867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb8a1e-f3d4-4581-926d-551107fa768c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:20.492539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb8a1e-f3d4-4581-926d-551107fa768c None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=7e572bf785284a8693cf386ced004524) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:20.492851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb8a1e-f3d4-4581-926d-551107fa768c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:20.494274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb8a1e-f3d4-4581-926d-551107fa768c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:20.507093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 344/682] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:20 2022] GET /identity/v3/role_assignments?user.id=7e572bf785284a8693cf386ced004524&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:20.507093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29722467-6510-4b31-953a-6dee2ff39303 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:20.507093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29722467-6510-4b31-953a-6dee2ff39303 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:20.507093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29722467-6510-4b31-953a-6dee2ff39303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:20.525212 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-29722467-6510-4b31-953a-6dee2ff39303 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:20.527430 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-29722467-6510-4b31-953a-6dee2ff39303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:20.533837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 339/683] 10.222.0.75 () {56 vars in 1135 bytes} [Thu Jul 14 10:54:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:20.538355 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bce00-b59f-4734-a692-46800b322c48 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:20.711012 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 345/684] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:20.720484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29722467-6510-4b31-953a-6dee2ff39303 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:20.742691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bce00-b59f-4734-a692-46800b322c48 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:20.800840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a11f5-e8cf-4c03-a504-37fa11c7e0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:20.801451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a11f5-e8cf-4c03-a504-37fa11c7e0ac None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:20.801740 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a11f5-e8cf-4c03-a504-37fa11c7e0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:20.802003 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a11f5-e8cf-4c03-a504-37fa11c7e0ac None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:20.802876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a11f5-e8cf-4c03-a504-37fa11c7e0ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:20.803188 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a11f5-e8cf-4c03-a504-37fa11c7e0ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:20.804677 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a11f5-e8cf-4c03-a504-37fa11c7e0ac None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:20.823646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 340/685] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:20.825612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a11f5-e8cf-4c03-a504-37fa11c7e0ac None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:20.851639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:20.861968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:20.908135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29722467-6510-4b31-953a-6dee2ff39303 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:21.004130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a11f5-e8cf-4c03-a504-37fa11c7e0ac None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:21.008634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f8f2e-2017-41d3-a56f-8b0bc2f94059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:21.009312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f8f2e-2017-41d3-a56f-8b0bc2f94059 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:21.009601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f8f2e-2017-41d3-a56f-8b0bc2f94059 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:21.009884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f8f2e-2017-41d3-a56f-8b0bc2f94059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:21.091108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba68f6a-3b4c-4062-b1f9-4e8061b61c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:21.092353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba68f6a-3b4c-4062-b1f9-4e8061b61c70 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:21.092649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba68f6a-3b4c-4062-b1f9-4e8061b61c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:21.092933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba68f6a-3b4c-4062-b1f9-4e8061b61c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:21.112544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba68f6a-3b4c-4062-b1f9-4e8061b61c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:21.114698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba68f6a-3b4c-4062-b1f9-4e8061b61c70 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:21.124646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f8f2e-2017-41d3-a56f-8b0bc2f94059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:21.126666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f8f2e-2017-41d3-a56f-8b0bc2f94059 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-7ba68f6a-3b4c-4062-b1f9-4e8061b61c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:21.133476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:21.134495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:21.136115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:21.136115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:21.136115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:21.136115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:21.136115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 341/686] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:21.136115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba68f6a-3b4c-4062-b1f9-4e8061b61c70 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:21.220518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:21.231226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:21.262107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 346/687] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:54:21.263531 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba68f6a-3b4c-4062-b1f9-4e8061b61c70 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:21.265870 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa4214-d2e9-4fe0-b1c4-11d3a6da278c None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:21.266161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa4214-d2e9-4fe0-b1c4-11d3a6da278c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:21.266441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa4214-d2e9-4fe0-b1c4-11d3a6da278c None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:21.274935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 347/688] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:21.274935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa4214-d2e9-4fe0-b1c4-11d3a6da278c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:21.308634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee7a07-578f-4713-a7c4-808185d68c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:21.309304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee7a07-578f-4713-a7c4-808185d68c20 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:21.309584 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee7a07-578f-4713-a7c4-808185d68c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:21.309861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee7a07-578f-4713-a7c4-808185d68c20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:21.310699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee7a07-578f-4713-a7c4-808185d68c20 None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=dadd485b7cac42f4aa079cab91973c35) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:21.311010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee7a07-578f-4713-a7c4-808185d68c20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:21.312521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee7a07-578f-4713-a7c4-808185d68c20 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:21.328064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 342/689] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:21 2022] GET /identity/v3/role_assignments?user.id=dadd485b7cac42f4aa079cab91973c35&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:21.328064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee7a07-578f-4713-a7c4-808185d68c20 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:21.439487 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa4214-d2e9-4fe0-b1c4-11d3a6da278c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:21.452262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee7a07-578f-4713-a7c4-808185d68c20 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:21.491568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:21.500829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d968d-ddf1-4916-8288-0611cefc41e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:21.501507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d968d-ddf1-4916-8288-0611cefc41e3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:21.501795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d968d-ddf1-4916-8288-0611cefc41e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:21.502074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d968d-ddf1-4916-8288-0611cefc41e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:21.502807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d968d-ddf1-4916-8288-0611cefc41e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:21.503122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d968d-ddf1-4916-8288-0611cefc41e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:21.509184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d968d-ddf1-4916-8288-0611cefc41e3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:21.509345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c4ce1-2a8a-495c-b6cb-094cefd68015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:21.509345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c4ce1-2a8a-495c-b6cb-094cefd68015 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:21.509345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c4ce1-2a8a-495c-b6cb-094cefd68015 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:21.509345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c4ce1-2a8a-495c-b6cb-094cefd68015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:21.520595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 348/690] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:21.524714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20eb42-dbde-42a8-92f3-bfda77f240d4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:21.525035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20eb42-dbde-42a8-92f3-bfda77f240d4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:21.525391 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20eb42-dbde-42a8-92f3-bfda77f240d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:21.526438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c4ce1-2a8a-495c-b6cb-094cefd68015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:21.528699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c4ce1-2a8a-495c-b6cb-094cefd68015 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-250c4ce1-2a8a-495c-b6cb-094cefd68015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:21.547675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:21.548624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:21.549515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:21.549515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:21.549515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:21.549515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:21.549770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 343/691] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:21.553320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c4ce1-2a8a-495c-b6cb-094cefd68015 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:21.556683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6c20eb42-dbde-42a8-92f3-bfda77f240d4 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:21.559110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6c20eb42-dbde-42a8-92f3-bfda77f240d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:21.594818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:21.643726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 349/692] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:21.644785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20eb42-dbde-42a8-92f3-bfda77f240d4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:21.675647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c4ce1-2a8a-495c-b6cb-094cefd68015 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:21.713762 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101ca1d-4ffa-41d3-9df6-8c50ed532592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:21.714363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101ca1d-4ffa-41d3-9df6-8c50ed532592 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:21.714711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101ca1d-4ffa-41d3-9df6-8c50ed532592 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:21.714994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101ca1d-4ffa-41d3-9df6-8c50ed532592 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:21.715782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101ca1d-4ffa-41d3-9df6-8c50ed532592 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:21.717256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101ca1d-4ffa-41d3-9df6-8c50ed532592 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:21.747515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 344/693] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:54:21 2022] POST /identity/v3/users => generated 254 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:54:21.748626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101ca1d-4ffa-41d3-9df6-8c50ed532592 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:21.818531 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:21.843416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20eb42-dbde-42a8-92f3-bfda77f240d4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:21.905495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544263c1-b93c-4b1c-9320-d8cfb43168fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:21.906195 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544263c1-b93c-4b1c-9320-d8cfb43168fe None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:21.906482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544263c1-b93c-4b1c-9320-d8cfb43168fe None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:21.906763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544263c1-b93c-4b1c-9320-d8cfb43168fe None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:21.907552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544263c1-b93c-4b1c-9320-d8cfb43168fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:21.907866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544263c1-b93c-4b1c-9320-d8cfb43168fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:21.909314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544263c1-b93c-4b1c-9320-d8cfb43168fe None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:21.927062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 350/694] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:21.931201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544263c1-b93c-4b1c-9320-d8cfb43168fe None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:21.966091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101ca1d-4ffa-41d3-9df6-8c50ed532592 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:21.987780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:22.022963 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b536f4-2006-44cd-a205-fe65b930d90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:22.023826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b536f4-2006-44cd-a205-fe65b930d90f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:22.024106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b536f4-2006-44cd-a205-fe65b930d90f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:22.024381 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b536f4-2006-44cd-a205-fe65b930d90f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:22.025155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b536f4-2006-44cd-a205-fe65b930d90f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:22.026847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b536f4-2006-44cd-a205-fe65b930d90f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-d4b536f4-2006-44cd-a205-fe65b930d90f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:22.045432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:22.046486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:22.047329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 345/695] 10.222.0.75 () {56 vars in 1129 bytes} [Thu Jul 14 10:54:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:22.053386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b536f4-2006-44cd-a205-fe65b930d90f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:22.103691 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544263c1-b93c-4b1c-9320-d8cfb43168fe None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:22.180732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd4ad7-d316-4f78-8594-50d91198122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:22.181672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd4ad7-d316-4f78-8594-50d91198122c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:22.181800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd4ad7-d316-4f78-8594-50d91198122c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:22.182114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd4ad7-d316-4f78-8594-50d91198122c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:22.199143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd4ad7-d316-4f78-8594-50d91198122c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:22.200937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd4ad7-d316-4f78-8594-50d91198122c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a9dd4ad7-d316-4f78-8594-50d91198122c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:22.221754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:22.223684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:22.225512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:22.225512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:22.225512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:22.225512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:22.225512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:22.225512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:22.225512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:22.225512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:22.225512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 351/696] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:22.236563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:22.240528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd4ad7-d316-4f78-8594-50d91198122c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:22.240974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b536f4-2006-44cd-a205-fe65b930d90f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:22.309424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c1755-bbf6-48c6-99af-9d0055d6f832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:22.310763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c1755-bbf6-48c6-99af-9d0055d6f832 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:22.311437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c1755-bbf6-48c6-99af-9d0055d6f832 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:22.312134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c1755-bbf6-48c6-99af-9d0055d6f832 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:22.313538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c1755-bbf6-48c6-99af-9d0055d6f832 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:22.314061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c1755-bbf6-48c6-99af-9d0055d6f832 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:22.315889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c1755-bbf6-48c6-99af-9d0055d6f832 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:22.322233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 346/697] 10.222.0.75 () {56 vars in 1124 bytes} [Thu Jul 14 10:54:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:22.327566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c1755-bbf6-48c6-99af-9d0055d6f832 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:22.385491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:22.406375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd4ad7-d316-4f78-8594-50d91198122c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:22.467823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1442d-612f-4af0-b628-292d07e73124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:22.468936 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1442d-612f-4af0-b628-292d07e73124 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:22.469415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1442d-612f-4af0-b628-292d07e73124 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:22.469888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1442d-612f-4af0-b628-292d07e73124 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:22.470890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1442d-612f-4af0-b628-292d07e73124 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:22.471456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1442d-612f-4af0-b628-292d07e73124 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:22.473276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1442d-612f-4af0-b628-292d07e73124 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:22.504310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 352/698] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:22.513223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1442d-612f-4af0-b628-292d07e73124 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:22.529656 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c1755-bbf6-48c6-99af-9d0055d6f832 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:22.626036 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12388e05-a479-455f-90c1-1b76e593dffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:22.627116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12388e05-a479-455f-90c1-1b76e593dffe None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:22.628191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12388e05-a479-455f-90c1-1b76e593dffe None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:22.629305 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12388e05-a479-455f-90c1-1b76e593dffe None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:22.631120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12388e05-a479-455f-90c1-1b76e593dffe None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:22.631685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12388e05-a479-455f-90c1-1b76e593dffe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:22.638041 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12388e05-a479-455f-90c1-1b76e593dffe None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:22.655416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 347/699] 10.222.0.75 () {56 vars in 1124 bytes} [Thu Jul 14 10:54:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:22.688134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1442d-612f-4af0-b628-292d07e73124 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:22.752169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a30a26-3b1b-4452-9428-85261163925f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:22.753258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a30a26-3b1b-4452-9428-85261163925f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:22.753761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a30a26-3b1b-4452-9428-85261163925f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:22.754231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a30a26-3b1b-4452-9428-85261163925f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:22.755938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a30a26-3b1b-4452-9428-85261163925f None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=978e71dfbb134f41871114824b03fbeb) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:22.756455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a30a26-3b1b-4452-9428-85261163925f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:22.758159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a30a26-3b1b-4452-9428-85261163925f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:22.770072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 353/700] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:22 2022] GET /identity/v3/role_assignments?user.id=978e71dfbb134f41871114824b03fbeb&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:22.941679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48df93-3012-4e87-8fd4-eecfb94f8f5d None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:22.942791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48df93-3012-4e87-8fd4-eecfb94f8f5d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:22.942791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48df93-3012-4e87-8fd4-eecfb94f8f5d None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:22.944495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 348/701] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:22.950520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcfa32-33d4-4aa1-9ec8-de5fe875e6a0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:22.950830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcfa32-33d4-4aa1-9ec8-de5fe875e6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:22.951102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcfa32-33d4-4aa1-9ec8-de5fe875e6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:22.970320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-affcfa32-33d4-4aa1-9ec8-de5fe875e6a0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:22.971610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-affcfa32-33d4-4aa1-9ec8-de5fe875e6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:22.974624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:23.064554 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 354/702] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:54:23.087755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48df93-3012-4e87-8fd4-eecfb94f8f5d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:23.125749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:23.225317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48df93-3012-4e87-8fd4-eecfb94f8f5d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:23.271019 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578a0ba-72bf-46f7-8336-edf174187dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:23.271869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578a0ba-72bf-46f7-8336-edf174187dac None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:23.272155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578a0ba-72bf-46f7-8336-edf174187dac None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:23.272430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578a0ba-72bf-46f7-8336-edf174187dac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:23.287933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578a0ba-72bf-46f7-8336-edf174187dac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-a578a0ba-72bf-46f7-8336-edf174187dac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:23.297287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:23.298240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:23.299103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:23.299103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:23.299103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:23.299103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:23.299103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:23.299103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:23.299103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:23.299103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:23.299103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:23.299103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:23.299103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:54:23.299103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:23.300478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 349/703] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:54:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:23.307476 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcfa32-33d4-4aa1-9ec8-de5fe875e6a0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:23.466285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcfa32-33d4-4aa1-9ec8-de5fe875e6a0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:23.513336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b5350-7562-4119-a7c7-80067f336fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:23.514332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b5350-7562-4119-a7c7-80067f336fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:23.514791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b5350-7562-4119-a7c7-80067f336fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:23.515240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b5350-7562-4119-a7c7-80067f336fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:23.516230 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b5350-7562-4119-a7c7-80067f336fc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:23.516753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b5350-7562-4119-a7c7-80067f336fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:23.518330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b5350-7562-4119-a7c7-80067f336fc2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:23.529258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 355/704] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:23.532454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:23.534601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578a0ba-72bf-46f7-8336-edf174187dac None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:23.642389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:23.663942 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578a0ba-72bf-46f7-8336-edf174187dac None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:23.700928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd285e05-1f9c-40ec-8a90-51f21699e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:23.701740 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd285e05-1f9c-40ec-8a90-51f21699e33a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:23.702028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd285e05-1f9c-40ec-8a90-51f21699e33a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:23.702307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd285e05-1f9c-40ec-8a90-51f21699e33a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:23.714589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd285e05-1f9c-40ec-8a90-51f21699e33a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:23.716091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd285e05-1f9c-40ec-8a90-51f21699e33a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-fd285e05-1f9c-40ec-8a90-51f21699e33a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:23.734427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:23.735340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:23.736256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:23.736256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:23.736256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:23.736256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:23.736256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 350/705] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:23.742488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b5350-7562-4119-a7c7-80067f336fc2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:23.865513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b5350-7562-4119-a7c7-80067f336fc2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:23.911060 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23135d-45ec-4f50-b45e-0c68d6e95c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:23.911773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23135d-45ec-4f50-b45e-0c68d6e95c8b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:23.912066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23135d-45ec-4f50-b45e-0c68d6e95c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:23.912347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23135d-45ec-4f50-b45e-0c68d6e95c8b None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:23.913086 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23135d-45ec-4f50-b45e-0c68d6e95c8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:23.913398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23135d-45ec-4f50-b45e-0c68d6e95c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:23.914774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23135d-45ec-4f50-b45e-0c68d6e95c8b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:23.934216 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 356/706] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:23.939752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd285e05-1f9c-40ec-8a90-51f21699e33a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:24.089200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd285e05-1f9c-40ec-8a90-51f21699e33a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:24.136020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd57e6a-7265-4703-815b-30ea73036e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:24.136833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd57e6a-7265-4703-815b-30ea73036e8a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:24.137119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd57e6a-7265-4703-815b-30ea73036e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:24.137398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd57e6a-7265-4703-815b-30ea73036e8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:24.150077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd57e6a-7265-4703-815b-30ea73036e8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:24.151569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd57e6a-7265-4703-815b-30ea73036e8a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:24.180008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-9cd57e6a-7265-4703-815b-30ea73036e8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:24.185000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:24.185946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:24.186870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:24.186870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:24.186870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:24.186870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:24.186870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:24.186870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:24.186870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:24.186870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:24.186870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:54:24.186870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:24.189771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 351/707] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Jul 14 10:54:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:24.201714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23135d-45ec-4f50-b45e-0c68d6e95c8b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:24.306553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:24.369741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23135d-45ec-4f50-b45e-0c68d6e95c8b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:24.420750 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84426405-900c-4908-88e3-054719170a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:24.421444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84426405-900c-4908-88e3-054719170a33 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:24.421735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84426405-900c-4908-88e3-054719170a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:24.422879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84426405-900c-4908-88e3-054719170a33 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:24.423802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84426405-900c-4908-88e3-054719170a33 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:24.424119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84426405-900c-4908-88e3-054719170a33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:24.425572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84426405-900c-4908-88e3-054719170a33 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:24.454856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 357/708] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Jul 14 10:54:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:24.460703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd57e6a-7265-4703-815b-30ea73036e8a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:24.474978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ff489-f0a9-4459-99ef-7de04ff5bc19 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:24.475305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ff489-f0a9-4459-99ef-7de04ff5bc19 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:24.475577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ff489-f0a9-4459-99ef-7de04ff5bc19 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:24.478418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 358/709] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:24.484959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1e449-251b-4ada-9cd4-7a6ce948a125 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:24.485277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1e449-251b-4ada-9cd4-7a6ce948a125 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:24.485540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1e449-251b-4ada-9cd4-7a6ce948a125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:24.516574 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-19f1e449-251b-4ada-9cd4-7a6ce948a125 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:24.517832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-19f1e449-251b-4ada-9cd4-7a6ce948a125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:24.579500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd57e6a-7265-4703-815b-30ea73036e8a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:24.622033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166327e-0080-4a38-98d1-553ca20faa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:24.622634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166327e-0080-4a38-98d1-553ca20faa83 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:24.622918 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166327e-0080-4a38-98d1-553ca20faa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:24.623195 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166327e-0080-4a38-98d1-553ca20faa83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:24.674327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 359/710] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:24.734696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:24.737568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166327e-0080-4a38-98d1-553ca20faa83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:24.740153 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166327e-0080-4a38-98d1-553ca20faa83 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:24.752936 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1e449-251b-4ada-9cd4-7a6ce948a125 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:24.847426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:24.854038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 352/711] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:54:24.865436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166327e-0080-4a38-98d1-553ca20faa83 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:24.915206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1e449-251b-4ada-9cd4-7a6ce948a125 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:24.962139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf4c81-013a-424c-b0de-554595d6a7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:24.963494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4c81-013a-424c-b0de-554595d6a7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:24.963975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4c81-013a-424c-b0de-554595d6a7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:24.964416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4c81-013a-424c-b0de-554595d6a7d2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:24.965417 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf4c81-013a-424c-b0de-554595d6a7d2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:24.967800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf4c81-013a-424c-b0de-554595d6a7d2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-bbaf4c81-013a-424c-b0de-554595d6a7d2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:24.983817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:24.984862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:24.985686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 360/712] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:24 2022] GET /identity/v3/services/network => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:24.987031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d14d6-9b15-48e6-b562-9b72cae6fb51 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:24.991407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d14d6-9b15-48e6-b562-9b72cae6fb51 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:24.991407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d14d6-9b15-48e6-b562-9b72cae6fb51 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:24.991407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 361/713] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:24.991407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d14d6-9b15-48e6-b562-9b72cae6fb51 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:25.029832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:25.033187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166327e-0080-4a38-98d1-553ca20faa83 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:25.072444 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9a33f-92f4-4fe1-87c8-26d69c4edc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:25.073256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9a33f-92f4-4fe1-87c8-26d69c4edc93 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.073543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9a33f-92f4-4fe1-87c8-26d69c4edc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.073825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9a33f-92f4-4fe1-87c8-26d69c4edc93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.085795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9a33f-92f4-4fe1-87c8-26d69c4edc93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:25.087368 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9a33f-92f4-4fe1-87c8-26d69c4edc93 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:25.103121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d14d6-9b15-48e6-b562-9b72cae6fb51 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-05d9a33f-92f4-4fe1-87c8-26d69c4edc93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:25.107343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:25.108325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:25.109327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:25.109327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:25.109327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:25.109327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:25.109327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 353/714] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:25.110259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39d5e9-0b07-4a21-adff-7fa5054e25ca None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.110549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39d5e9-0b07-4a21-adff-7fa5054e25ca None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.110827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39d5e9-0b07-4a21-adff-7fa5054e25ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.128520 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9c39d5e9-0b07-4a21-adff-7fa5054e25ca None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:25.129778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9c39d5e9-0b07-4a21-adff-7fa5054e25ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:25.145760 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:25.149221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04ba87-d680-4488-866a-0b3903910846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:25.150026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04ba87-d680-4488-866a-0b3903910846 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.150316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04ba87-d680-4488-866a-0b3903910846 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.150661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04ba87-d680-4488-866a-0b3903910846 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.151548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04ba87-d680-4488-866a-0b3903910846 None admin] RBAC: Adding query filter params (name=network) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:25.151962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04ba87-d680-4488-866a-0b3903910846 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:25.153502 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04ba87-d680-4488-866a-0b3903910846 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:25.159625 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 362/715] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:54:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:25.161374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04ba87-d680-4488-866a-0b3903910846 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:25.235477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 354/716] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:25.236765 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39d5e9-0b07-4a21-adff-7fa5054e25ca None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:25.284754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04ba87-d680-4488-866a-0b3903910846 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:25.336432 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d589c01-ac2f-4d6d-af7e-38861219f8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:25.337627 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d589c01-ac2f-4d6d-af7e-38861219f8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.338095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d589c01-ac2f-4d6d-af7e-38861219f8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.338551 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d589c01-ac2f-4d6d-af7e-38861219f8d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.339490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d589c01-ac2f-4d6d-af7e-38861219f8d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:25.340033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d589c01-ac2f-4d6d-af7e-38861219f8d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:25.341642 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d589c01-ac2f-4d6d-af7e-38861219f8d3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:25.354717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 363/717] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:25.355944 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d589c01-ac2f-4d6d-af7e-38861219f8d3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:25.356054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:25.379187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39d5e9-0b07-4a21-adff-7fa5054e25ca None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:25.403658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:25.421612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2146418-8c4f-4972-9b37-c471601afff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:25.422293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2146418-8c4f-4972-9b37-c471601afff8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.422581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2146418-8c4f-4972-9b37-c471601afff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.422863 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2146418-8c4f-4972-9b37-c471601afff8 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.423604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2146418-8c4f-4972-9b37-c471601afff8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:25.423919 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2146418-8c4f-4972-9b37-c471601afff8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:25.425313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2146418-8c4f-4972-9b37-c471601afff8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:25.430371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 355/718] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:54:25 2022] GET /identity/v3/services?type=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:25.431204 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2146418-8c4f-4972-9b37-c471601afff8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:25.510626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d589c01-ac2f-4d6d-af7e-38861219f8d3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:25.563481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb79261f-ef11-48f7-b169-7497bcfddb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:25.565011 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79261f-ef11-48f7-b169-7497bcfddb68 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.565537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79261f-ef11-48f7-b169-7497bcfddb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.566314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79261f-ef11-48f7-b169-7497bcfddb68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.582453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb79261f-ef11-48f7-b169-7497bcfddb68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-bb79261f-ef11-48f7-b169-7497bcfddb68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:25.594489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:25.595577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:25.596437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:25.596437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:25.596437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:25.596437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:25.596437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:25.596437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:25.596437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:25.596437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:25.596437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:25.596437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:25.596437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:25.596437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:25.597078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 364/719] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:25.598210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb75764-b990-48e8-9367-8f829cc7a794 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.598615 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb75764-b990-48e8-9367-8f829cc7a794 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.598981 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2146418-8c4f-4972-9b37-c471601afff8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:25.599435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb75764-b990-48e8-9367-8f829cc7a794 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.604931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 365/720] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:25.604931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb75764-b990-48e8-9367-8f829cc7a794 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:25.662713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc15d9-de27-48db-99df-5d182539eb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:25.663814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc15d9-de27-48db-99df-5d182539eb62 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.664852 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc15d9-de27-48db-99df-5d182539eb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.665319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc15d9-de27-48db-99df-5d182539eb62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.675746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc15d9-de27-48db-99df-5d182539eb62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:25.678378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc15d9-de27-48db-99df-5d182539eb62 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-81bc15d9-de27-48db-99df-5d182539eb62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:25.700797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:25.701947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:25.702996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:25.702996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:25.702996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:25.702996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:25.702996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:25.702996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:25.702996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:25.702996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:25.710275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 356/721] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:25.710275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f627d-41fa-4946-8766-0089999c2f01 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.710275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f627d-41fa-4946-8766-0089999c2f01 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.710275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f627d-41fa-4946-8766-0089999c2f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.724656 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:25.732099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f23f627d-41fa-4946-8766-0089999c2f01 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:25.733108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:25.733850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f23f627d-41fa-4946-8766-0089999c2f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:25.741295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb75764-b990-48e8-9367-8f829cc7a794 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:25.782917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad84d5-9800-4fe0-9e58-2775c2ffbc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:25.783681 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad84d5-9800-4fe0-9e58-2775c2ffbc72 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.783991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad84d5-9800-4fe0-9e58-2775c2ffbc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.784250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad84d5-9800-4fe0-9e58-2775c2ffbc72 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.785126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad84d5-9800-4fe0-9e58-2775c2ffbc72 None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:25.785450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad84d5-9800-4fe0-9e58-2775c2ffbc72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:25.787076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad84d5-9800-4fe0-9e58-2775c2ffbc72 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:25.797460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 366/722] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:25.803409 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad84d5-9800-4fe0-9e58-2775c2ffbc72 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:25.819407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 357/723] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:25.819407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced61ad7-5095-48e4-937d-004bf68c0544 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.819407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced61ad7-5095-48e4-937d-004bf68c0544 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.819407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced61ad7-5095-48e4-937d-004bf68c0544 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.819407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 358/724] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:25.819407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced61ad7-5095-48e4-937d-004bf68c0544 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:25.940226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad84d5-9800-4fe0-9e58-2775c2ffbc72 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:25.950858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced61ad7-5095-48e4-937d-004bf68c0544 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:25.988320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b1a1b-af81-4066-ab6c-04eb5a31769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:25.989656 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b1a1b-af81-4066-ab6c-04eb5a31769f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.989949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b1a1b-af81-4066-ab6c-04eb5a31769f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.990232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b1a1b-af81-4066-ab6c-04eb5a31769f None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.991043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b1a1b-af81-4066-ab6c-04eb5a31769f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:25.991363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b1a1b-af81-4066-ab6c-04eb5a31769f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:25.992871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b1a1b-af81-4066-ab6c-04eb5a31769f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:25.993039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e2742-fa9e-456f-849e-7787f4cbfe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:25.993843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e2742-fa9e-456f-849e-7787f4cbfe05 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:25.994133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e2742-fa9e-456f-849e-7787f4cbfe05 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:25.994410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e2742-fa9e-456f-849e-7787f4cbfe05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:25.997744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:26.007400 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 367/725] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:26.009751 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61566b50-0436-47f1-b90c-1f3bad95a6da None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:26.010249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61566b50-0436-47f1-b90c-1f3bad95a6da None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:26.010532 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61566b50-0436-47f1-b90c-1f3bad95a6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:26.012346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e2742-fa9e-456f-849e-7787f4cbfe05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:26.014916 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e2742-fa9e-456f-849e-7787f4cbfe05 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:26.027814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-61566b50-0436-47f1-b90c-1f3bad95a6da None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:26.029018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-61566b50-0436-47f1-b90c-1f3bad95a6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:26.043543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-603e2742-fa9e-456f-849e-7787f4cbfe05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:26.044063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:26.045035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:26.045912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:26.045912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:26.045912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:26.045912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:26.045912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 359/726] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:26.046187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e2742-fa9e-456f-849e-7787f4cbfe05 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:26.120532 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 368/727] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:26.121706 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61566b50-0436-47f1-b90c-1f3bad95a6da None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:26.187742 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e2742-fa9e-456f-849e-7787f4cbfe05 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:26.228850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48d8de-e038-4bf2-bdb0-171c2364a8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:26.229526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48d8de-e038-4bf2-bdb0-171c2364a8e9 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:26.229811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48d8de-e038-4bf2-bdb0-171c2364a8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:26.230086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48d8de-e038-4bf2-bdb0-171c2364a8e9 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:26.230806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48d8de-e038-4bf2-bdb0-171c2364a8e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:26.232217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48d8de-e038-4bf2-bdb0-171c2364a8e9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:26.251425 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 360/728] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:54:26 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:54:26.251425 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48d8de-e038-4bf2-bdb0-171c2364a8e9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:26.276730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61566b50-0436-47f1-b90c-1f3bad95a6da None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:26.323015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:26.356516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f5d71-44de-4346-b22c-7c33be977e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:26.357581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f5d71-44de-4346-b22c-7c33be977e8d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:26.358113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f5d71-44de-4346-b22c-7c33be977e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:26.358590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f5d71-44de-4346-b22c-7c33be977e8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:26.359813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f5d71-44de-4346-b22c-7c33be977e8d None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=7e572bf785284a8693cf386ced004524) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:26.361008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f5d71-44de-4346-b22c-7c33be977e8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:26.363113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f5d71-44de-4346-b22c-7c33be977e8d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:26.372522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 369/729] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:26 2022] GET /identity/v3/role_assignments?user.id=7e572bf785284a8693cf386ced004524&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:26.373685 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f5d71-44de-4346-b22c-7c33be977e8d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:26.387888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:26.422711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48d8de-e038-4bf2-bdb0-171c2364a8e9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:26.477212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f594765-2479-4718-8f0f-31fe10e2121c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:26.478246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f594765-2479-4718-8f0f-31fe10e2121c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:26.478536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f594765-2479-4718-8f0f-31fe10e2121c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:26.478809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f594765-2479-4718-8f0f-31fe10e2121c None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:26.479788 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f594765-2479-4718-8f0f-31fe10e2121c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:26.480094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f594765-2479-4718-8f0f-31fe10e2121c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:26.482263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f594765-2479-4718-8f0f-31fe10e2121c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:26.500702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 361/730] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:26.506193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f594765-2479-4718-8f0f-31fe10e2121c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:26.544879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f5d71-44de-4346-b22c-7c33be977e8d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:26.601054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3b90d-ade3-4d63-ba9f-5e6c10425a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:26.602258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3b90d-ade3-4d63-ba9f-5e6c10425a8d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:26.603023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3b90d-ade3-4d63-ba9f-5e6c10425a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:26.604050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3b90d-ade3-4d63-ba9f-5e6c10425a8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:26.621195 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3b90d-ade3-4d63-ba9f-5e6c10425a8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-3fd3b90d-ade3-4d63-ba9f-5e6c10425a8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:26.632138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:26.633135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:26.633964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:26.633964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:26.633964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:26.633964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:26.633964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:26.633964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:26.633964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:26.633964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:26.633964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:26.633964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:26.633964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:54:26.633964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:26.637439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 370/731] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:54:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:26.643813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3b90d-ade3-4d63-ba9f-5e6c10425a8d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:26.676116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f594765-2479-4718-8f0f-31fe10e2121c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:26.697846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:26.750990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab0e16-ac16-4f44-bc9d-7683b58f8c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:26.751847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab0e16-ac16-4f44-bc9d-7683b58f8c5c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:26.752126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab0e16-ac16-4f44-bc9d-7683b58f8c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:26.752404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab0e16-ac16-4f44-bc9d-7683b58f8c5c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:26.770582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab0e16-ac16-4f44-bc9d-7683b58f8c5c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:26.772085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab0e16-ac16-4f44-bc9d-7683b58f8c5c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:26.783237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f3ab0e16-ac16-4f44-bc9d-7683b58f8c5c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:26.803540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:26.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:26.805462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:26.805462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:26.805462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:26.805462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:26.805462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:26.805462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:26.805462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:26.805462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:26.805462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:54:26.805462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:26.811848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 362/732] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Jul 14 10:54:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:26.817355 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab0e16-ac16-4f44-bc9d-7683b58f8c5c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:26.826548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3b90d-ade3-4d63-ba9f-5e6c10425a8d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:26.869626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0c687-3ad3-42c5-a8e1-9ca966cf8f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:26.870653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0c687-3ad3-42c5-a8e1-9ca966cf8f21 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:26.871156 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0c687-3ad3-42c5-a8e1-9ca966cf8f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:26.871653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0c687-3ad3-42c5-a8e1-9ca966cf8f21 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:26.872613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0c687-3ad3-42c5-a8e1-9ca966cf8f21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:26.873132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0c687-3ad3-42c5-a8e1-9ca966cf8f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:26.874895 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0c687-3ad3-42c5-a8e1-9ca966cf8f21 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:26.892498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 371/733] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:26.899310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0c687-3ad3-42c5-a8e1-9ca966cf8f21 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:27.008545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab0e16-ac16-4f44-bc9d-7683b58f8c5c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:27.079142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fad72-e157-4ff5-898d-178ebe874545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:27.079871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fad72-e157-4ff5-898d-178ebe874545 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:27.080161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fad72-e157-4ff5-898d-178ebe874545 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:27.080444 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fad72-e157-4ff5-898d-178ebe874545 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:27.081304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105fad72-e157-4ff5-898d-178ebe874545 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:27.081617 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105fad72-e157-4ff5-898d-178ebe874545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:27.083683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105fad72-e157-4ff5-898d-178ebe874545 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:27.122490 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 363/734] 10.222.0.75 () {56 vars in 1149 bytes} [Thu Jul 14 10:54:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:27.129016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fad72-e157-4ff5-898d-178ebe874545 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:27.208159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:27.225922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0c687-3ad3-42c5-a8e1-9ca966cf8f21 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:27.255964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:27.265819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abc900-f498-41a8-8c9e-e161e7a13701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:27.266630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abc900-f498-41a8-8c9e-e161e7a13701 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:27.266913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abc900-f498-41a8-8c9e-e161e7a13701 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:27.267185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abc900-f498-41a8-8c9e-e161e7a13701 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:27.286920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abc900-f498-41a8-8c9e-e161e7a13701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:27.288437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8abc900-f498-41a8-8c9e-e161e7a13701 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-f8abc900-f498-41a8-8c9e-e161e7a13701 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:27.326963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:27.327983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:27.328958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:27.328958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:27.328958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:27.328958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:27.328958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 372/735] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:27.335165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abc900-f498-41a8-8c9e-e161e7a13701 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:27.352921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fad72-e157-4ff5-898d-178ebe874545 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:27.424908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cd281-8870-4f0e-90fc-fea8406d79df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:27.425581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cd281-8870-4f0e-90fc-fea8406d79df None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:27.425863 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cd281-8870-4f0e-90fc-fea8406d79df None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:27.426135 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cd281-8870-4f0e-90fc-fea8406d79df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:27.533535 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abc900-f498-41a8-8c9e-e161e7a13701 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:27.561510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cd281-8870-4f0e-90fc-fea8406d79df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:27.562986 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cd281-8870-4f0e-90fc-fea8406d79df None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:27.595695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fcfd0-0227-4e9c-aad0-803f4ba71223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:27.596389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fcfd0-0227-4e9c-aad0-803f4ba71223 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:27.596662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fcfd0-0227-4e9c-aad0-803f4ba71223 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:27.596942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fcfd0-0227-4e9c-aad0-803f4ba71223 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:27.597682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fcfd0-0227-4e9c-aad0-803f4ba71223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:27.597993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fcfd0-0227-4e9c-aad0-803f4ba71223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:27.600370 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fcfd0-0227-4e9c-aad0-803f4ba71223 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:27.623335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:27.624801 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 373/736] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:27.632449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fcfd0-0227-4e9c-aad0-803f4ba71223 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:27.692487 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:27.707273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 364/737] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:27 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:54:27.713768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f56a4-40ae-4111-a420-612a67855121 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:27.714234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f56a4-40ae-4111-a420-612a67855121 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:27.714526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f56a4-40ae-4111-a420-612a67855121 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:27.716230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 365/738] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:27.717005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f56a4-40ae-4111-a420-612a67855121 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:27.752601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fcfd0-0227-4e9c-aad0-803f4ba71223 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:27.789572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda9674-2560-452c-a82e-108afabe7816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:27.790777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda9674-2560-452c-a82e-108afabe7816 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:27.791278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda9674-2560-452c-a82e-108afabe7816 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:27.791845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda9674-2560-452c-a82e-108afabe7816 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:27.800036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda9674-2560-452c-a82e-108afabe7816 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:27.802209 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda9674-2560-452c-a82e-108afabe7816 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-0cda9674-2560-452c-a82e-108afabe7816 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:27.818620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:27.820083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:27.821016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:27.821016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:27.821016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:27.821016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:27.821016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:27.821016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:27.821016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:27.821016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:27.821016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:54:27.821016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:27.823523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 374/739] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:54:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:27.825066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f041a-68d9-4bce-9363-995ec66e7c53 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:27.825572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f041a-68d9-4bce-9363-995ec66e7c53 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:27.826058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f041a-68d9-4bce-9363-995ec66e7c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:27.836812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f56a4-40ae-4111-a420-612a67855121 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:27.844923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a55f041a-68d9-4bce-9363-995ec66e7c53 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:27.846603 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a55f041a-68d9-4bce-9363-995ec66e7c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:27.876732 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e9f7e-f773-408c-89b7-8fcf265ccd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:27.877554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e9f7e-f773-408c-89b7-8fcf265ccd4c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:27.877930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e9f7e-f773-408c-89b7-8fcf265ccd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:27.878126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e9f7e-f773-408c-89b7-8fcf265ccd4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:27.887780 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:27.890014 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e9f7e-f773-408c-89b7-8fcf265ccd4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:27.891676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e9f7e-f773-408c-89b7-8fcf265ccd4c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-247e9f7e-f773-408c-89b7-8fcf265ccd4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:27.910821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:27.912025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:27.913130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:27.913130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:27.913130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:27.913130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:27.916958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 366/740] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:27.917922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e9f7e-f773-408c-89b7-8fcf265ccd4c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:27.927305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 375/741] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:54:27.928621 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f041a-68d9-4bce-9363-995ec66e7c53 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:27.967986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:28.029257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e9f7e-f773-408c-89b7-8fcf265ccd4c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:28.044145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f041a-68d9-4bce-9363-995ec66e7c53 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:28.065416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314484ef-e3f4-4d2a-ad98-107996d9446c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:28.066107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314484ef-e3f4-4d2a-ad98-107996d9446c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:28.066395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314484ef-e3f4-4d2a-ad98-107996d9446c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:28.066676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314484ef-e3f4-4d2a-ad98-107996d9446c None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:28.068122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314484ef-e3f4-4d2a-ad98-107996d9446c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:28.068437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314484ef-e3f4-4d2a-ad98-107996d9446c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:28.069903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314484ef-e3f4-4d2a-ad98-107996d9446c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:28.084879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7e786-8e0e-4fc7-8d95-17d3fb9ac9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:28.086010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e786-8e0e-4fc7-8d95-17d3fb9ac9dc None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:28.086440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e786-8e0e-4fc7-8d95-17d3fb9ac9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:28.086857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e786-8e0e-4fc7-8d95-17d3fb9ac9dc None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:28.087801 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7e786-8e0e-4fc7-8d95-17d3fb9ac9dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:28.088300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7e786-8e0e-4fc7-8d95-17d3fb9ac9dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:28.089900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7e786-8e0e-4fc7-8d95-17d3fb9ac9dc None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:28.093960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 367/742] 10.222.0.75 () {56 vars in 1135 bytes} [Thu Jul 14 10:54:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:28.094720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314484ef-e3f4-4d2a-ad98-107996d9446c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:28.105013 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 376/743] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:28.106148 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7e786-8e0e-4fc7-8d95-17d3fb9ac9dc None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:28.151349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:28.245111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:28.251932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314484ef-e3f4-4d2a-ad98-107996d9446c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:28.257878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7e786-8e0e-4fc7-8d95-17d3fb9ac9dc None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:28.300401 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a47b1-7d55-41ac-970a-ea9044c00cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:28.301288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a47b1-7d55-41ac-970a-ea9044c00cae None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:28.301582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a47b1-7d55-41ac-970a-ea9044c00cae None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:28.301865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a47b1-7d55-41ac-970a-ea9044c00cae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:28.309748 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff89af2-bcb0-43e4-a651-1e3f824ffb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:28.310782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff89af2-bcb0-43e4-a651-1e3f824ffb00 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:28.311286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff89af2-bcb0-43e4-a651-1e3f824ffb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:28.311806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff89af2-bcb0-43e4-a651-1e3f824ffb00 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:28.312871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff89af2-bcb0-43e4-a651-1e3f824ffb00 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:28.314685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff89af2-bcb0-43e4-a651-1e3f824ffb00 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:28.343235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 368/744] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:54:28 2022] POST /identity/v3/services => generated 240 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:54:28.344583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff89af2-bcb0-43e4-a651-1e3f824ffb00 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:28.384613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a47b1-7d55-41ac-970a-ea9044c00cae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:28.386050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a47b1-7d55-41ac-970a-ea9044c00cae None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:28.488870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:28.503084 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff89af2-bcb0-43e4-a651-1e3f824ffb00 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:28.507709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:28.519871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 377/745] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:28 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:54:28.527356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a47b1-7d55-41ac-970a-ea9044c00cae None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:28.565538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0ede9-f756-4ebb-90b3-f9253340889b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:28.566521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0ede9-f756-4ebb-90b3-f9253340889b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:28.567965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0ede9-f756-4ebb-90b3-f9253340889b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:28.567965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0ede9-f756-4ebb-90b3-f9253340889b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:28.579193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0ede9-f756-4ebb-90b3-f9253340889b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:28.582228 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0ede9-f756-4ebb-90b3-f9253340889b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-afe0ede9-f756-4ebb-90b3-f9253340889b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:28.603085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:28.604292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:28.605326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:28.605326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:28.605326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:28.605326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:28.605326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:28.605326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:28.605326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:28.605326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:28.606458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 369/746] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:28.612228 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0ede9-f756-4ebb-90b3-f9253340889b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:28.653167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a47b1-7d55-41ac-970a-ea9044c00cae None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:28.695587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcdf39-4d22-4aae-a4fc-e36b186c4db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:28.697410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcdf39-4d22-4aae-a4fc-e36b186c4db0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:28.698171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcdf39-4d22-4aae-a4fc-e36b186c4db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:28.698903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcdf39-4d22-4aae-a4fc-e36b186c4db0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:28.712213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcdf39-4d22-4aae-a4fc-e36b186c4db0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:28.714061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcdf39-4d22-4aae-a4fc-e36b186c4db0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-e8bcdf39-4d22-4aae-a4fc-e36b186c4db0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:28.743007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:28.743990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:28.744871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:28.744871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:28.744871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:28.744871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:28.744871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 378/747] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:28.744871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcdf39-4d22-4aae-a4fc-e36b186c4db0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:28.747300 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0ede9-f756-4ebb-90b3-f9253340889b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:28.787825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c8520-c332-410d-aef9-d5f3d99ca3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:28.788430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c8520-c332-410d-aef9-d5f3d99ca3ea None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:28.788720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c8520-c332-410d-aef9-d5f3d99ca3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:28.789063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c8520-c332-410d-aef9-d5f3d99ca3ea None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:28.790006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c8520-c332-410d-aef9-d5f3d99ca3ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:28.790365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c8520-c332-410d-aef9-d5f3d99ca3ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:28.791869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c8520-c332-410d-aef9-d5f3d99ca3ea None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:28.796924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:28.809285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 370/748] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:28.815106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c8520-c332-410d-aef9-d5f3d99ca3ea None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:28.831776 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:28.893525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcdf39-4d22-4aae-a4fc-e36b186c4db0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:28.935772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c8520-c332-410d-aef9-d5f3d99ca3ea None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:28.938660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf3ec9-bf41-4bb9-b1ef-6c32c8b494ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:28.940150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf3ec9-bf41-4bb9-b1ef-6c32c8b494ea None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:28.940150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf3ec9-bf41-4bb9-b1ef-6c32c8b494ea None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:28.940150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf3ec9-bf41-4bb9-b1ef-6c32c8b494ea None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:28.940710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf3ec9-bf41-4bb9-b1ef-6c32c8b494ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:28.941014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf3ec9-bf41-4bb9-b1ef-6c32c8b494ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:28.943027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf3ec9-bf41-4bb9-b1ef-6c32c8b494ea None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:28.958771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 379/749] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:28.958771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f4f40-df2a-4ec8-891f-ceff77fc90d7 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:28.958771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f4f40-df2a-4ec8-891f-ceff77fc90d7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:28.958771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f4f40-df2a-4ec8-891f-ceff77fc90d7 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:28.960132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 380/750] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:28.965615 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7b7e1-9cd8-4e5b-a0a9-fb7724f58f14 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:28.966177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7b7e1-9cd8-4e5b-a0a9-fb7724f58f14 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:28.966355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7b7e1-9cd8-4e5b-a0a9-fb7724f58f14 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:28.967886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 381/751] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:28.972355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7b7e1-9cd8-4e5b-a0a9-fb7724f58f14 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:28.983450 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef983bc-f149-427a-98ee-b93dd38aa204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:28.984159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef983bc-f149-427a-98ee-b93dd38aa204 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:28.984472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef983bc-f149-427a-98ee-b93dd38aa204 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:28.984752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef983bc-f149-427a-98ee-b93dd38aa204 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:28.985924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef983bc-f149-427a-98ee-b93dd38aa204 None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=f40c0fcc93cf4bbf8d35c1de70d087ba) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:28.986425 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef983bc-f149-427a-98ee-b93dd38aa204 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:28.989624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef983bc-f149-427a-98ee-b93dd38aa204 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:29.007419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 371/752] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:28 2022] GET /identity/v3/role_assignments?user.id=f40c0fcc93cf4bbf8d35c1de70d087ba&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:29.007419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25129824-3e20-4458-83c8-ff39c4bca304 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:29.007419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25129824-3e20-4458-83c8-ff39c4bca304 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:29.007419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25129824-3e20-4458-83c8-ff39c4bca304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:29.031835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-25129824-3e20-4458-83c8-ff39c4bca304 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:29.033202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-25129824-3e20-4458-83c8-ff39c4bca304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:29.115323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7b7e1-9cd8-4e5b-a0a9-fb7724f58f14 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:29.119246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:29.134374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 372/753] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:29.136118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758be58-8b19-4550-a833-f3b294b17bb9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:29.136556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758be58-8b19-4550-a833-f3b294b17bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:29.136974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758be58-8b19-4550-a833-f3b294b17bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:29.161768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:29.171751 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e758be58-8b19-4550-a833-f3b294b17bb9 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:29.173623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e758be58-8b19-4550-a833-f3b294b17bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:29.174778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d03f1-5a8d-4c95-9a9c-85ed2e29a412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:29.175737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d03f1-5a8d-4c95-9a9c-85ed2e29a412 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:29.176116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d03f1-5a8d-4c95-9a9c-85ed2e29a412 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:29.177452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d03f1-5a8d-4c95-9a9c-85ed2e29a412 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:29.200958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d03f1-5a8d-4c95-9a9c-85ed2e29a412 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:29.207407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d03f1-5a8d-4c95-9a9c-85ed2e29a412 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-450d03f1-5a8d-4c95-9a9c-85ed2e29a412 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:29.217703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:29.218641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:29.219560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:29.219560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:29.219560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:29.219560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:29.219560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:29.219560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:29.219560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:29.219560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:29.219560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 382/754] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:29.220616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d03f1-5a8d-4c95-9a9c-85ed2e29a412 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:29.254541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 373/755] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:29.256183 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758be58-8b19-4550-a833-f3b294b17bb9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:29.334877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d03f1-5a8d-4c95-9a9c-85ed2e29a412 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:29.372455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e5144-d0dc-4ee9-b5d7-3baf1cebfcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:29.373290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e5144-d0dc-4ee9-b5d7-3baf1cebfcd3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:29.373586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e5144-d0dc-4ee9-b5d7-3baf1cebfcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:29.373866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e5144-d0dc-4ee9-b5d7-3baf1cebfcd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:29.385856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e5144-d0dc-4ee9-b5d7-3baf1cebfcd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:29.390089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758be58-8b19-4550-a833-f3b294b17bb9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-ec6e5144-d0dc-4ee9-b5d7-3baf1cebfcd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:29.394044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:29.396844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 383/756] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:54:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:29.398847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e5144-d0dc-4ee9-b5d7-3baf1cebfcd3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:29.426444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:29.431497 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4e10c-bf4e-4d70-a286-676fe66c706e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:29.433872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4e10c-bf4e-4d70-a286-676fe66c706e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:29.434317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4e10c-bf4e-4d70-a286-676fe66c706e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:29.434740 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4e10c-bf4e-4d70-a286-676fe66c706e None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:29.435712 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4e10c-bf4e-4d70-a286-676fe66c706e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:29.436285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4e10c-bf4e-4d70-a286-676fe66c706e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:29.438025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4e10c-bf4e-4d70-a286-676fe66c706e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:29.441078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:29.454107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 374/757] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:29.455898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4e10c-bf4e-4d70-a286-676fe66c706e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:29.536296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e5144-d0dc-4ee9-b5d7-3baf1cebfcd3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:29.565450 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4e10c-bf4e-4d70-a286-676fe66c706e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:29.579730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a25d2-c4cc-44fa-913d-db8c784ddc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:29.580862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a25d2-c4cc-44fa-913d-db8c784ddc96 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:29.581451 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a25d2-c4cc-44fa-913d-db8c784ddc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:29.581937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a25d2-c4cc-44fa-913d-db8c784ddc96 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:29.582878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a25d2-c4cc-44fa-913d-db8c784ddc96 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:29.584489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a25d2-c4cc-44fa-913d-db8c784ddc96 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-838a25d2-c4cc-44fa-913d-db8c784ddc96 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:29.597960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:29.599110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:54:29.600229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:29.602890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 384/758] 10.222.0.75 () {56 vars in 1129 bytes} [Thu Jul 14 10:54:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:29.604024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a25d2-c4cc-44fa-913d-db8c784ddc96 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:29.606390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595965a-321b-42da-aff5-001805f01289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:29.607184 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595965a-321b-42da-aff5-001805f01289 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:29.607502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595965a-321b-42da-aff5-001805f01289 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:29.607784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595965a-321b-42da-aff5-001805f01289 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:29.609541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595965a-321b-42da-aff5-001805f01289 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:29.609541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595965a-321b-42da-aff5-001805f01289 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:29.610634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595965a-321b-42da-aff5-001805f01289 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:29.621684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 375/759] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:29.622821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595965a-321b-42da-aff5-001805f01289 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:29.696857 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:29.706331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:29.719831 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a25d2-c4cc-44fa-913d-db8c784ddc96 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:29.753212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595965a-321b-42da-aff5-001805f01289 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:29.758458 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6ad7f-9edc-4c84-b815-f86275ab900c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:29.759132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6ad7f-9edc-4c84-b815-f86275ab900c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:29.759445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6ad7f-9edc-4c84-b815-f86275ab900c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:29.759733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6ad7f-9edc-4c84-b815-f86275ab900c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:29.760583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6ad7f-9edc-4c84-b815-f86275ab900c None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=978e71dfbb134f41871114824b03fbeb) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:29.760899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6ad7f-9edc-4c84-b815-f86275ab900c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:29.762350 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6ad7f-9edc-4c84-b815-f86275ab900c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:29.769013 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 385/760] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:29 2022] GET /identity/v3/role_assignments?user.id=978e71dfbb134f41871114824b03fbeb&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:29.769863 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6ad7f-9edc-4c84-b815-f86275ab900c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:29.801967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782aacdc-ad11-4fcb-af26-47e07a3af5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:29.803159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782aacdc-ad11-4fcb-af26-47e07a3af5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:29.803733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782aacdc-ad11-4fcb-af26-47e07a3af5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:29.804210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782aacdc-ad11-4fcb-af26-47e07a3af5a9 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:29.805126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782aacdc-ad11-4fcb-af26-47e07a3af5a9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:29.805653 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782aacdc-ad11-4fcb-af26-47e07a3af5a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:29.807449 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782aacdc-ad11-4fcb-af26-47e07a3af5a9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:29.813927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 376/761] 10.222.0.75 () {56 vars in 1124 bytes} [Thu Jul 14 10:54:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:29.821752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782aacdc-ad11-4fcb-af26-47e07a3af5a9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:29.890398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6ad7f-9edc-4c84-b815-f86275ab900c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:29.928858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e3910-a1d9-4b8a-bedc-3b9bf83f291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:29.929676 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e3910-a1d9-4b8a-bedc-3b9bf83f291b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:29.929963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e3910-a1d9-4b8a-bedc-3b9bf83f291b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:29.930243 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e3910-a1d9-4b8a-bedc-3b9bf83f291b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:29.942628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e3910-a1d9-4b8a-bedc-3b9bf83f291b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:29.944143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e3910-a1d9-4b8a-bedc-3b9bf83f291b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-151e3910-a1d9-4b8a-bedc-3b9bf83f291b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:29.965846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:29.967548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:29.973950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:29.973950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:29.973950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:29.973950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:29.973950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 386/762] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:29.973950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e3910-a1d9-4b8a-bedc-3b9bf83f291b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:29.975601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:29.988159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782aacdc-ad11-4fcb-af26-47e07a3af5a9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:30.046291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:30.053688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652d030-f431-4602-a546-350c634fe98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:30.055214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652d030-f431-4602-a546-350c634fe98a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:30.056037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652d030-f431-4602-a546-350c634fe98a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:30.056734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652d030-f431-4602-a546-350c634fe98a None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:30.058446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652d030-f431-4602-a546-350c634fe98a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:30.059108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652d030-f431-4602-a546-350c634fe98a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:30.062076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652d030-f431-4602-a546-350c634fe98a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:30.074025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 377/763] 10.222.0.75 () {56 vars in 1124 bytes} [Thu Jul 14 10:54:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:30.075768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652d030-f431-4602-a546-350c634fe98a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:30.086687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e3910-a1d9-4b8a-bedc-3b9bf83f291b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:30.123148 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be3852-5c77-4f0e-92d2-c84e313b1b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:30.124420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be3852-5c77-4f0e-92d2-c84e313b1b28 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:30.124420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be3852-5c77-4f0e-92d2-c84e313b1b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:30.124819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be3852-5c77-4f0e-92d2-c84e313b1b28 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:30.126165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be3852-5c77-4f0e-92d2-c84e313b1b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:30.126324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be3852-5c77-4f0e-92d2-c84e313b1b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:30.127980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be3852-5c77-4f0e-92d2-c84e313b1b28 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:30.141325 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 387/764] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:30.147466 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be3852-5c77-4f0e-92d2-c84e313b1b28 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:30.216226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:30.256471 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652d030-f431-4602-a546-350c634fe98a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:30.266892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be3852-5c77-4f0e-92d2-c84e313b1b28 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:30.298183 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2967aad-bb86-43c4-bec4-faa6e081ff37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:30.298858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2967aad-bb86-43c4-bec4-faa6e081ff37 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:30.299142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2967aad-bb86-43c4-bec4-faa6e081ff37 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:30.299553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2967aad-bb86-43c4-bec4-faa6e081ff37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:30.300458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2967aad-bb86-43c4-bec4-faa6e081ff37 None admin] RBAC: Adding query filter params (interface=public, service_id=d1c49b1d974a41de9022ba94aad3d359, region_id=RegionOne) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:30.300778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2967aad-bb86-43c4-bec4-faa6e081ff37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:30.302286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2967aad-bb86-43c4-bec4-faa6e081ff37 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:30.311953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e182cf-d078-467f-a472-3283eb33dad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:30.313444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e182cf-d078-467f-a472-3283eb33dad2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:30.313734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e182cf-d078-467f-a472-3283eb33dad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:30.314014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e182cf-d078-467f-a472-3283eb33dad2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:30.323883 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e182cf-d078-467f-a472-3283eb33dad2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:30.325401 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e182cf-d078-467f-a472-3283eb33dad2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:30.331565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 378/765] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Jul 14 10:54:30 2022] GET /identity/v3/endpoints?service_id=d1c49b1d974a41de9022ba94aad3d359&interface=public®ion_id=RegionOne => generated 194 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:30.340620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2967aad-bb86-43c4-bec4-faa6e081ff37 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c0e182cf-d078-467f-a472-3283eb33dad2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:30.347249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:30.348829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:30.349782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:30.349782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:30.349782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:30.349782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:30.349782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:30.349782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:30.349782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:30.349782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:30.349782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:54:30.349782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:30.350786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 388/766] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Jul 14 10:54:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:30.357877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e182cf-d078-467f-a472-3283eb33dad2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:30.431850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:30.469824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2967aad-bb86-43c4-bec4-faa6e081ff37 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:30.508625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403275d0-4376-4956-9d44-15a972b13a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:30.509384 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403275d0-4376-4956-9d44-15a972b13a94 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:30.509684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403275d0-4376-4956-9d44-15a972b13a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:30.509972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403275d0-4376-4956-9d44-15a972b13a94 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:30.510678 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403275d0-4376-4956-9d44-15a972b13a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:30.512170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403275d0-4376-4956-9d44-15a972b13a94 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:30.523637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 379/767] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Jul 14 10:54:30 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:30.615676 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e182cf-d078-467f-a472-3283eb33dad2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:30.647807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:30.672714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3b04b-0d31-4ad3-845f-c5728d58d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:30.673691 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3b04b-0d31-4ad3-845f-c5728d58d1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:30.673977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3b04b-0d31-4ad3-845f-c5728d58d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:30.674254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3b04b-0d31-4ad3-845f-c5728d58d1e5 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:30.675694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3b04b-0d31-4ad3-845f-c5728d58d1e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:30.676238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3b04b-0d31-4ad3-845f-c5728d58d1e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:30.677759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3b04b-0d31-4ad3-845f-c5728d58d1e5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:30.700393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 389/768] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Jul 14 10:54:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:30.706623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403275d0-4376-4956-9d44-15a972b13a94 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:30.933783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403275d0-4376-4956-9d44-15a972b13a94 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:30.949717 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:30.972511 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c70fc-edee-4806-9e11-5bf9f016b553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:30.973364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c70fc-edee-4806-9e11-5bf9f016b553 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:30.973649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c70fc-edee-4806-9e11-5bf9f016b553 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:30.973932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c70fc-edee-4806-9e11-5bf9f016b553 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:30.985258 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c70fc-edee-4806-9e11-5bf9f016b553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:30.986761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c70fc-edee-4806-9e11-5bf9f016b553 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f02c70fc-edee-4806-9e11-5bf9f016b553 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:31.014397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:31.015338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:31.016273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:31.016273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:31.016273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:31.016273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:31.016273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 380/769] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:31.021564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3b04b-0d31-4ad3-845f-c5728d58d1e5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:31.024470 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f6c21-83ad-45eb-a87a-14a77e9912a7 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:31.024761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f6c21-83ad-45eb-a87a-14a77e9912a7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:31.025047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f6c21-83ad-45eb-a87a-14a77e9912a7 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:31.026686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 381/770] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:31.032068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92110da4-de40-464a-8bbf-2df708df1f33 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:31.032352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92110da4-de40-464a-8bbf-2df708df1f33 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:31.032632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92110da4-de40-464a-8bbf-2df708df1f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:31.049544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-92110da4-de40-464a-8bbf-2df708df1f33 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:31.050762 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-92110da4-de40-464a-8bbf-2df708df1f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:31.170838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3b04b-0d31-4ad3-845f-c5728d58d1e5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:31.227868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b16d3b-65a2-48a7-be95-9817f54cfa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:31.228778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b16d3b-65a2-48a7-be95-9817f54cfa37 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:31.228890 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 382/771] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:54:31.229357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b16d3b-65a2-48a7-be95-9817f54cfa37 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:31.229800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b16d3b-65a2-48a7-be95-9817f54cfa37 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:31.235413 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b16d3b-65a2-48a7-be95-9817f54cfa37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:31.236106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b16d3b-65a2-48a7-be95-9817f54cfa37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:31.237707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b16d3b-65a2-48a7-be95-9817f54cfa37 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:31.255431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 390/772] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:31.263427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92110da4-de40-464a-8bbf-2df708df1f33 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:31.269799 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b16d3b-65a2-48a7-be95-9817f54cfa37 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:31.330378 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:31.401659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b16d3b-65a2-48a7-be95-9817f54cfa37 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:31.446021 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ece6a3-5b48-45a1-a59c-391b1788d6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:31.447226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ece6a3-5b48-45a1-a59c-391b1788d6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:31.447740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ece6a3-5b48-45a1-a59c-391b1788d6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:31.448230 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ece6a3-5b48-45a1-a59c-391b1788d6c7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:31.449462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ece6a3-5b48-45a1-a59c-391b1788d6c7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:31.451904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ece6a3-5b48-45a1-a59c-391b1788d6c7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-d9ece6a3-5b48-45a1-a59c-391b1788d6c7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:31.472395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:31.473972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:54:31.474819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:31.476660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 391/773] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:31 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:31.483988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ece6a3-5b48-45a1-a59c-391b1788d6c7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:31.561891 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:31.577840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92110da4-de40-464a-8bbf-2df708df1f33 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:31.613772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ece6a3-5b48-45a1-a59c-391b1788d6c7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:31.622365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:31.623541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617d753-2c2b-49d7-a6df-f1831d4cf926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:31.624432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617d753-2c2b-49d7-a6df-f1831d4cf926 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:31.624751 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617d753-2c2b-49d7-a6df-f1831d4cf926 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:31.625547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617d753-2c2b-49d7-a6df-f1831d4cf926 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:31.636596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617d753-2c2b-49d7-a6df-f1831d4cf926 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:31.638639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617d753-2c2b-49d7-a6df-f1831d4cf926 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:31.654732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef403b-929f-423d-8343-261553b583ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:31.655459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef403b-929f-423d-8343-261553b583ed None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:31.655798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef403b-929f-423d-8343-261553b583ed None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:31.656085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef403b-929f-423d-8343-261553b583ed None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-9617d753-2c2b-49d7-a6df-f1831d4cf926 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:31.656804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:31.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:31.658563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:31.658563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:31.658563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:31.658563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:31.658563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:31.658563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:31.658563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:31.658563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:31.658978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef403b-929f-423d-8343-261553b583ed None admin] RBAC: Adding query filter params (name=network) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:31.658978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef403b-929f-423d-8343-261553b583ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:31.659112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 383/774] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:31.659178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef403b-929f-423d-8343-261553b583ed None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:31.660278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4aa51-34ed-4570-96d6-6e2b96a3f894 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:31.660278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4aa51-34ed-4570-96d6-6e2b96a3f894 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:31.660278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4aa51-34ed-4570-96d6-6e2b96a3f894 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:31.662492 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 384/775] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:31.666558 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4aa51-34ed-4570-96d6-6e2b96a3f894 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:31.669614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 392/776] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:54:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:31.671047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2886f29-15e2-4f28-bf4a-ae91355cba5f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:31.671435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2886f29-15e2-4f28-bf4a-ae91355cba5f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:31.671807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2886f29-15e2-4f28-bf4a-ae91355cba5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:31.691662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f2886f29-15e2-4f28-bf4a-ae91355cba5f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:31.692934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f2886f29-15e2-4f28-bf4a-ae91355cba5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:31.787519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 393/777] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:31.787519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2886f29-15e2-4f28-bf4a-ae91355cba5f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:31.827615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4aa51-34ed-4570-96d6-6e2b96a3f894 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:31.872199 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e8fb2-95a9-4280-863b-c73ea3f58fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:31.873257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e8fb2-95a9-4280-863b-c73ea3f58fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:31.873753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e8fb2-95a9-4280-863b-c73ea3f58fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:31.874262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e8fb2-95a9-4280-863b-c73ea3f58fd1 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:31.875321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e8fb2-95a9-4280-863b-c73ea3f58fd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:31.875924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e8fb2-95a9-4280-863b-c73ea3f58fd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:31.877826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e8fb2-95a9-4280-863b-c73ea3f58fd1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:31.897246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 385/778] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:31.898673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e8fb2-95a9-4280-863b-c73ea3f58fd1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:31.923893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:31.932635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:31.937080 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2886f29-15e2-4f28-bf4a-ae91355cba5f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:31.989995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7780f4e-5755-482a-882f-a8aa738572e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:31.990701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7780f4e-5755-482a-882f-a8aa738572e4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:31.991205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7780f4e-5755-482a-882f-a8aa738572e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:31.991552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7780f4e-5755-482a-882f-a8aa738572e4 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:31.992274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7780f4e-5755-482a-882f-a8aa738572e4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:31.992584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7780f4e-5755-482a-882f-a8aa738572e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:31.994776 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7780f4e-5755-482a-882f-a8aa738572e4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:32.003293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 394/779] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:54:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:32.005672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7780f4e-5755-482a-882f-a8aa738572e4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:32.047504 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e8fb2-95a9-4280-863b-c73ea3f58fd1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:32.090675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51159b21-149c-4174-9000-94d659a2fe18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:32.091990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51159b21-149c-4174-9000-94d659a2fe18 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:32.092627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51159b21-149c-4174-9000-94d659a2fe18 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:32.093153 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51159b21-149c-4174-9000-94d659a2fe18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:32.115644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51159b21-149c-4174-9000-94d659a2fe18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-51159b21-149c-4174-9000-94d659a2fe18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:32.124273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:32.125252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:32.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 386/780] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:32.126753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51159b21-149c-4174-9000-94d659a2fe18 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:32.144445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7780f4e-5755-482a-882f-a8aa738572e4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:32.185391 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5ec9c-f8d0-47a7-9fe7-2e3614863d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:32.186797 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5ec9c-f8d0-47a7-9fe7-2e3614863d0a None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:32.187419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5ec9c-f8d0-47a7-9fe7-2e3614863d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:32.187912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5ec9c-f8d0-47a7-9fe7-2e3614863d0a None admin] PATH_INFO: `/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/7e572bf785284a8693cf386ced004524/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:32.194971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5ec9c-f8d0-47a7-9fe7-2e3614863d0a None admin] RBAC: Authorizing `identity:create_grant(project_id=a68601f16d1c49548b6e3af0b9819ba3, user_id=7e572bf785284a8693cf386ced004524, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:32.197872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5ec9c-f8d0-47a7-9fe7-2e3614863d0a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:32.219456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 395/781] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:54:32 2022] PUT /identity/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/7e572bf785284a8693cf386ced004524/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:54:32.220613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5ec9c-f8d0-47a7-9fe7-2e3614863d0a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:32.264561 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:32.295370 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:32.308560 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51159b21-149c-4174-9000-94d659a2fe18 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:32.363399 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ba8ee-51b0-476a-9110-5e32a269bb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:32.364711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ba8ee-51b0-476a-9110-5e32a269bb01 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:32.365047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ba8ee-51b0-476a-9110-5e32a269bb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:32.365330 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ba8ee-51b0-476a-9110-5e32a269bb01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:32.366150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ba8ee-51b0-476a-9110-5e32a269bb01 None admin] RBAC: Adding query filter params (interface=public, service_id=c782d88af3c84603bb7d182ebd26c377, region_id=RegionOne) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:32.366466 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ba8ee-51b0-476a-9110-5e32a269bb01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:32.368190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ba8ee-51b0-476a-9110-5e32a269bb01 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:32.378173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 387/782] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Jul 14 10:54:32 2022] GET /identity/v3/endpoints?service_id=c782d88af3c84603bb7d182ebd26c377&interface=public®ion_id=RegionOne => generated 194 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:32.380443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45805855-dc7b-4a1a-83a8-2fbd06fdf3d9 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:32.380560 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45805855-dc7b-4a1a-83a8-2fbd06fdf3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:32.380772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45805855-dc7b-4a1a-83a8-2fbd06fdf3d9 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:32.386627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 388/783] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:32.386627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45805855-dc7b-4a1a-83a8-2fbd06fdf3d9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:32.425299 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5ec9c-f8d0-47a7-9fe7-2e3614863d0a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:32.468315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0151785-cab5-4a0a-b5af-3638b45c9d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:32.469915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0151785-cab5-4a0a-b5af-3638b45c9d17 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:32.470786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0151785-cab5-4a0a-b5af-3638b45c9d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:32.471482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0151785-cab5-4a0a-b5af-3638b45c9d17 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:32.473026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0151785-cab5-4a0a-b5af-3638b45c9d17 None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:32.473775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0151785-cab5-4a0a-b5af-3638b45c9d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:32.476099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0151785-cab5-4a0a-b5af-3638b45c9d17 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:32.486063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 396/784] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:32.487711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea92f5-ea13-499f-90f9-cce85ef76a2b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:32.488212 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea92f5-ea13-499f-90f9-cce85ef76a2b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:32.489265 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea92f5-ea13-499f-90f9-cce85ef76a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:32.507974 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-cbea92f5-ea13-499f-90f9-cce85ef76a2b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:32.509659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cbea92f5-ea13-499f-90f9-cce85ef76a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:32.514320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45805855-dc7b-4a1a-83a8-2fbd06fdf3d9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:32.571025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff764d5-8042-4dfd-8a4d-6d6f31c1469a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:32.571661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff764d5-8042-4dfd-8a4d-6d6f31c1469a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:32.571945 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff764d5-8042-4dfd-8a4d-6d6f31c1469a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:32.572223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff764d5-8042-4dfd-8a4d-6d6f31c1469a None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:32.572991 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff764d5-8042-4dfd-8a4d-6d6f31c1469a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:32.574436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff764d5-8042-4dfd-8a4d-6d6f31c1469a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:32.580402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 389/785] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Jul 14 10:54:32 2022] GET /identity/v3/services => generated 772 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:32.583428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff764d5-8042-4dfd-8a4d-6d6f31c1469a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:32.611808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:32.630339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 397/786] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:32.689686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea92f5-ea13-499f-90f9-cce85ef76a2b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:32.730547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:32.770692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff764d5-8042-4dfd-8a4d-6d6f31c1469a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:32.826888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91a9c2-c8d2-4c83-afcf-9fd031875ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:32.827737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91a9c2-c8d2-4c83-afcf-9fd031875ccd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:32.828021 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91a9c2-c8d2-4c83-afcf-9fd031875ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:32.828295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91a9c2-c8d2-4c83-afcf-9fd031875ccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:32.840663 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91a9c2-c8d2-4c83-afcf-9fd031875ccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:32.842148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91a9c2-c8d2-4c83-afcf-9fd031875ccd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-0f91a9c2-c8d2-4c83-afcf-9fd031875ccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:32.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:32.864773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:32.866162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:32.866162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:32.866162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:32.866162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:32.866546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea92f5-ea13-499f-90f9-cce85ef76a2b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:32.866677 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 390/787] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:32.872844 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91a9c2-c8d2-4c83-afcf-9fd031875ccd None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:32.901847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bda011-3e0d-4264-91c9-c89f80607543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:32.902661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bda011-3e0d-4264-91c9-c89f80607543 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:32.902946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bda011-3e0d-4264-91c9-c89f80607543 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:32.903225 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bda011-3e0d-4264-91c9-c89f80607543 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:32.918892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bda011-3e0d-4264-91c9-c89f80607543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-83bda011-3e0d-4264-91c9-c89f80607543 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:32.929657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:32.930654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:32.932349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:32.932349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:32.932349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:32.932349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:32.932349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:32.932349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:32.932349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:32.932349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:32.932349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:32.932349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:32.932349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:54:32.932349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:32.935140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 398/788] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:54:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:32.946180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bda011-3e0d-4264-91c9-c89f80607543 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:32.950181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:33.038888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:33.049613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91a9c2-c8d2-4c83-afcf-9fd031875ccd None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:33.074585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bda011-3e0d-4264-91c9-c89f80607543 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:33.099846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1298e-1791-4bde-ab65-7364cea8d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:33.100438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1298e-1791-4bde-ab65-7364cea8d8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.100720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1298e-1791-4bde-ab65-7364cea8d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.100998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1298e-1791-4bde-ab65-7364cea8d8c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.101971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1298e-1791-4bde-ab65-7364cea8d8c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:33.102095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1298e-1791-4bde-ab65-7364cea8d8c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:33.103444 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1298e-1791-4bde-ab65-7364cea8d8c4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:33.120434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 391/789] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:33.120434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbca06d-412b-47ef-ad38-9a8d1f0df84a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.120434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbca06d-412b-47ef-ad38-9a8d1f0df84a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.120434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbca06d-412b-47ef-ad38-9a8d1f0df84a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.120434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 392/790] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:33.124988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbca06d-412b-47ef-ad38-9a8d1f0df84a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:33.125535 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dad179-49a5-4785-835f-18847df99e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:33.126401 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dad179-49a5-4785-835f-18847df99e22 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.126838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dad179-49a5-4785-835f-18847df99e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.127270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dad179-49a5-4785-835f-18847df99e22 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.128503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dad179-49a5-4785-835f-18847df99e22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:33.129934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dad179-49a5-4785-835f-18847df99e22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:33.131635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dad179-49a5-4785-835f-18847df99e22 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:33.143855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 399/791] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:33.146085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a35bb7-91a4-40f6-ba49-ad0a428549b7 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.147073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a35bb7-91a4-40f6-ba49-ad0a428549b7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.147531 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a35bb7-91a4-40f6-ba49-ad0a428549b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.168693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-73a35bb7-91a4-40f6-ba49-ad0a428549b7 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:33.170254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-73a35bb7-91a4-40f6-ba49-ad0a428549b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:33.257886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbca06d-412b-47ef-ad38-9a8d1f0df84a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:33.268703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 400/792] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:33.270388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a35bb7-91a4-40f6-ba49-ad0a428549b7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:33.296576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:33.325962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ba689-5a5a-4865-b53c-83fa44a91e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:33.326810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ba689-5a5a-4865-b53c-83fa44a91e37 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.327097 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ba689-5a5a-4865-b53c-83fa44a91e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.327372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ba689-5a5a-4865-b53c-83fa44a91e37 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.337787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ba689-5a5a-4865-b53c-83fa44a91e37 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:33.339734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ba689-5a5a-4865-b53c-83fa44a91e37 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-d85ba689-5a5a-4865-b53c-83fa44a91e37 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:33.361754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:33.362997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:33.364237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:33.364237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:33.364237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:33.364237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:33.364237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:33.364237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:33.364237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:33.364237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:33.364237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:54:33.364237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:33.367458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 393/793] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Jul 14 10:54:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:33.369086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ba689-5a5a-4865-b53c-83fa44a91e37 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:33.377143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:33.433609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a35bb7-91a4-40f6-ba49-ad0a428549b7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:33.483318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b8a2f-9257-4e3f-b3b6-e48ab281a193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:33.484616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b8a2f-9257-4e3f-b3b6-e48ab281a193 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.485045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b8a2f-9257-4e3f-b3b6-e48ab281a193 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.485471 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b8a2f-9257-4e3f-b3b6-e48ab281a193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.500811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b8a2f-9257-4e3f-b3b6-e48ab281a193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:33.502559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b8a2f-9257-4e3f-b3b6-e48ab281a193 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:33.527350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ba689-5a5a-4865-b53c-83fa44a91e37 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c12b8a2f-9257-4e3f-b3b6-e48ab281a193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:33.527590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:33.528679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:33.529679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:33.529679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:33.529679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:33.529679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:33.529679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 401/794] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:33.529979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b8a2f-9257-4e3f-b3b6-e48ab281a193 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:33.585631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406aca2-1be4-4969-9d30-a2e58092191f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:33.586754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406aca2-1be4-4969-9d30-a2e58092191f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.587212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406aca2-1be4-4969-9d30-a2e58092191f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.587709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406aca2-1be4-4969-9d30-a2e58092191f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.588592 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406aca2-1be4-4969-9d30-a2e58092191f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:33.590187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406aca2-1be4-4969-9d30-a2e58092191f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-0406aca2-1be4-4969-9d30-a2e58092191f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:33.603992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:33.605154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:54:33.605985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:33.608711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 394/795] 10.222.0.75 () {56 vars in 1129 bytes} [Thu Jul 14 10:54:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:33.610889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406aca2-1be4-4969-9d30-a2e58092191f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:33.660458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:33.665473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b8a2f-9257-4e3f-b3b6-e48ab281a193 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:33.691489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:33.709939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c4c05-c270-45da-9fa2-d9e8f305de61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:33.711202 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c4c05-c270-45da-9fa2-d9e8f305de61 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.711530 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c4c05-c270-45da-9fa2-d9e8f305de61 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.711820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c4c05-c270-45da-9fa2-d9e8f305de61 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.712678 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c4c05-c270-45da-9fa2-d9e8f305de61 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:33.712997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c4c05-c270-45da-9fa2-d9e8f305de61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:33.714444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c4c05-c270-45da-9fa2-d9e8f305de61 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:33.742293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 402/796] 10.222.0.75 () {56 vars in 1149 bytes} [Thu Jul 14 10:54:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:33.742293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c4c05-c270-45da-9fa2-d9e8f305de61 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:33.744274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406aca2-1be4-4969-9d30-a2e58092191f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:33.787745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523e86a-715c-4086-85bb-f21047838c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:33.788483 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523e86a-715c-4086-85bb-f21047838c3d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.788781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523e86a-715c-4086-85bb-f21047838c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.789062 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523e86a-715c-4086-85bb-f21047838c3d None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.789804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523e86a-715c-4086-85bb-f21047838c3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:33.790111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523e86a-715c-4086-85bb-f21047838c3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:33.791529 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523e86a-715c-4086-85bb-f21047838c3d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:33.807310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 395/797] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:33.808547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523e86a-715c-4086-85bb-f21047838c3d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:33.875803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c4c05-c270-45da-9fa2-d9e8f305de61 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:33.917410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a3106-4f07-41e6-8de4-cb5d1995ad84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:33.918380 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a3106-4f07-41e6-8de4-cb5d1995ad84 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.918815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a3106-4f07-41e6-8de4-cb5d1995ad84 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.919263 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a3106-4f07-41e6-8de4-cb5d1995ad84 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.920177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a3106-4f07-41e6-8de4-cb5d1995ad84 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:33.920633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a3106-4f07-41e6-8de4-cb5d1995ad84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:33.922155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a3106-4f07-41e6-8de4-cb5d1995ad84 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:33.928321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 403/798] 10.222.0.75 () {56 vars in 1124 bytes} [Thu Jul 14 10:54:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:33.929512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a3106-4f07-41e6-8de4-cb5d1995ad84 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:33.951889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:33.956076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523e86a-715c-4086-85bb-f21047838c3d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:33.990962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:34.000799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd797d51-7fb7-4e9c-bfd4-8b3c775daa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:34.001962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd797d51-7fb7-4e9c-bfd4-8b3c775daa95 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:34.002438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd797d51-7fb7-4e9c-bfd4-8b3c775daa95 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:34.002862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd797d51-7fb7-4e9c-bfd4-8b3c775daa95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:34.016365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd797d51-7fb7-4e9c-bfd4-8b3c775daa95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:34.018113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd797d51-7fb7-4e9c-bfd4-8b3c775daa95 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-bd797d51-7fb7-4e9c-bfd4-8b3c775daa95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:34.038837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:34.040035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:34.040962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:34.040962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:34.040962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:34.040962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:34.047445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 396/799] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:34.048738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd797d51-7fb7-4e9c-bfd4-8b3c775daa95 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:34.059271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a3106-4f07-41e6-8de4-cb5d1995ad84 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:34.103613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb430a-4f34-46f8-8e01-78df58008ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:34.104531 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb430a-4f34-46f8-8e01-78df58008ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:34.104913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb430a-4f34-46f8-8e01-78df58008ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:34.105696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb430a-4f34-46f8-8e01-78df58008ea0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:34.116470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb430a-4f34-46f8-8e01-78df58008ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:34.118496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb430a-4f34-46f8-8e01-78df58008ea0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a8fb430a-4f34-46f8-8e01-78df58008ea0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:34.139068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:34.140335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:34.141495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:34.141495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:34.141495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:34.141495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:34.141495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:34.141495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:34.141495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:34.141495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:34.141495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:54:34.141495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:34.142913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 404/800] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:54:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:34.143923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb430a-4f34-46f8-8e01-78df58008ea0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:34.181114 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd797d51-7fb7-4e9c-bfd4-8b3c775daa95 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:34.241601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c1694-5f0c-4174-9450-19abec5c5405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:34.242430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c1694-5f0c-4174-9450-19abec5c5405 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:34.242806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c1694-5f0c-4174-9450-19abec5c5405 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:34.243180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c1694-5f0c-4174-9450-19abec5c5405 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:34.244022 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c1694-5f0c-4174-9450-19abec5c5405 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:34.244422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c1694-5f0c-4174-9450-19abec5c5405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:34.245895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c1694-5f0c-4174-9450-19abec5c5405 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:34.251767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 397/801] 10.222.0.75 () {56 vars in 1124 bytes} [Thu Jul 14 10:54:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:34.252818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c1694-5f0c-4174-9450-19abec5c5405 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:34.271886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:34.311933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb430a-4f34-46f8-8e01-78df58008ea0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:34.314146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:34.351585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656db004-ce00-45ef-873c-174773054ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:34.352521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656db004-ce00-45ef-873c-174773054ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:34.353028 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656db004-ce00-45ef-873c-174773054ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:34.353543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656db004-ce00-45ef-873c-174773054ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:34.354502 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656db004-ce00-45ef-873c-174773054ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:34.354998 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656db004-ce00-45ef-873c-174773054ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:34.356734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656db004-ce00-45ef-873c-174773054ae6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:34.375902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 405/802] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:34.377191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656db004-ce00-45ef-873c-174773054ae6 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:34.387199 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c1694-5f0c-4174-9450-19abec5c5405 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:34.432571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d31144-fd75-4c00-bde7-1d4ff9261a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:34.433477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d31144-fd75-4c00-bde7-1d4ff9261a8e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:34.433761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d31144-fd75-4c00-bde7-1d4ff9261a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:34.434037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d31144-fd75-4c00-bde7-1d4ff9261a8e None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:34.435480 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d31144-fd75-4c00-bde7-1d4ff9261a8e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:34.436004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d31144-fd75-4c00-bde7-1d4ff9261a8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:34.438580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d31144-fd75-4c00-bde7-1d4ff9261a8e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:34.463577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 398/803] 10.222.0.75 () {56 vars in 1135 bytes} [Thu Jul 14 10:54:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:34.464516 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d31144-fd75-4c00-bde7-1d4ff9261a8e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:34.506076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656db004-ce00-45ef-873c-174773054ae6 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:34.548068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6125154-67d9-4012-abcb-c2a8b9fee8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:34.550815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6125154-67d9-4012-abcb-c2a8b9fee8e7 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:34.550815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6125154-67d9-4012-abcb-c2a8b9fee8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:34.550815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6125154-67d9-4012-abcb-c2a8b9fee8e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:34.550815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6125154-67d9-4012-abcb-c2a8b9fee8e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:34.552373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6125154-67d9-4012-abcb-c2a8b9fee8e7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:34.556536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:34.578099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 406/804] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Jul 14 10:54:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:54:34.579862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf3991-f140-48cd-9378-3101f60b09c8 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:34.580162 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf3991-f140-48cd-9378-3101f60b09c8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:34.580449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf3991-f140-48cd-9378-3101f60b09c8 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:34.581945 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 407/805] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:34.583134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf3991-f140-48cd-9378-3101f60b09c8 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:34.607459 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d31144-fd75-4c00-bde7-1d4ff9261a8e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:34.612401 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:34.661677 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d39b4b-148e-44ec-8906-7f81748bb074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:34.662996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d39b4b-148e-44ec-8906-7f81748bb074 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:34.663518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d39b4b-148e-44ec-8906-7f81748bb074 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:34.664164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d39b4b-148e-44ec-8906-7f81748bb074 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:34.677020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d39b4b-148e-44ec-8906-7f81748bb074 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:34.679310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d39b4b-148e-44ec-8906-7f81748bb074 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-95d39b4b-148e-44ec-8906-7f81748bb074 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:34.697536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:34.698460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:34.699323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:34.699323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:34.699323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:34.699323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:34.699323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:34.699323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:34.699323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:34.699323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:34.706997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 399/806] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:34.709503 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6db006-2386-4640-94f9-1151ff0f292f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:34.710119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6db006-2386-4640-94f9-1151ff0f292f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:34.710652 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6db006-2386-4640-94f9-1151ff0f292f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:34.735818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf3991-f140-48cd-9378-3101f60b09c8 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:34.736858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ca6db006-2386-4640-94f9-1151ff0f292f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:34.738427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ca6db006-2386-4640-94f9-1151ff0f292f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:34.784897 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77846f-5035-4136-964e-43d6562e6a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:34.786305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77846f-5035-4136-964e-43d6562e6a5e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:34.786587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77846f-5035-4136-964e-43d6562e6a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:34.786861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77846f-5035-4136-964e-43d6562e6a5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:34.800224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77846f-5035-4136-964e-43d6562e6a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:34.802188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77846f-5035-4136-964e-43d6562e6a5e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-fd77846f-5035-4136-964e-43d6562e6a5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:34.823557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:34.824498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:34.825395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:34.825395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:34.825395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:34.825395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:34.825395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 408/807] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:34.825670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77846f-5035-4136-964e-43d6562e6a5e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:34.856798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 400/808] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:34.857690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6db006-2386-4640-94f9-1151ff0f292f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:34.930922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:34.955861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:34.990029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77846f-5035-4136-964e-43d6562e6a5e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:34.994036 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6db006-2386-4640-94f9-1151ff0f292f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:35.027945 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3023b-e487-4ea2-89d7-f3f6f5378c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:35.029058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3023b-e487-4ea2-89d7-f3f6f5378c5a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:35.029058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3023b-e487-4ea2-89d7-f3f6f5378c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:35.029231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3023b-e487-4ea2-89d7-f3f6f5378c5a None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:35.029995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3023b-e487-4ea2-89d7-f3f6f5378c5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:35.030336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3023b-e487-4ea2-89d7-f3f6f5378c5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:35.031798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3023b-e487-4ea2-89d7-f3f6f5378c5a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:35.040804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ff3cc-4f27-445e-b1a7-8c52c17afd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:35.041844 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ff3cc-4f27-445e-b1a7-8c52c17afd22 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:35.042234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ff3cc-4f27-445e-b1a7-8c52c17afd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:35.042538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ff3cc-4f27-445e-b1a7-8c52c17afd22 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:35.043250 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ff3cc-4f27-445e-b1a7-8c52c17afd22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:35.043601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ff3cc-4f27-445e-b1a7-8c52c17afd22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:35.045008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ff3cc-4f27-445e-b1a7-8c52c17afd22 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:35.049852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 409/809] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:35.049852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3023b-e487-4ea2-89d7-f3f6f5378c5a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:35.067332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 401/810] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:35.067332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf6656-4dd1-443b-9a78-29a30f2b379e None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:35.067332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf6656-4dd1-443b-9a78-29a30f2b379e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:35.067332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf6656-4dd1-443b-9a78-29a30f2b379e None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:35.067332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 402/811] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:35.067332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf6656-4dd1-443b-9a78-29a30f2b379e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:35.188627 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3023b-e487-4ea2-89d7-f3f6f5378c5a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:35.191096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf6656-4dd1-443b-9a78-29a30f2b379e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:35.218918 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:35.228256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:35.234054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becc1ef-1f57-4605-81e5-b33604fb3cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:35.234521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd89bb-830d-4bf5-97c4-37ea24f938d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:35.235033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becc1ef-1f57-4605-81e5-b33604fb3cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:35.235412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becc1ef-1f57-4605-81e5-b33604fb3cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:35.235515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd89bb-830d-4bf5-97c4-37ea24f938d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:35.235707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd89bb-830d-4bf5-97c4-37ea24f938d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:35.235808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becc1ef-1f57-4605-81e5-b33604fb3cd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:35.236000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd89bb-830d-4bf5-97c4-37ea24f938d7 None admin] PATH_INFO: `/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/f40c0fcc93cf4bbf8d35c1de70d087ba/roles/be88bd87b07244debee4e9545284ff13` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:35.248924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5becc1ef-1f57-4605-81e5-b33604fb3cd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:35.253094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd89bb-830d-4bf5-97c4-37ea24f938d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a68601f16d1c49548b6e3af0b9819ba3, user_id=f40c0fcc93cf4bbf8d35c1de70d087ba, role_id=be88bd87b07244debee4e9545284ff13)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:35.255322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd89bb-830d-4bf5-97c4-37ea24f938d7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-5becc1ef-1f57-4605-81e5-b33604fb3cd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:35.257178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:35.258103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:35.258951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 410/812] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:54:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:35.259683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becc1ef-1f57-4605-81e5-b33604fb3cd6 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:35.278286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 403/813] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:54:35 2022] PUT /identity/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/f40c0fcc93cf4bbf8d35c1de70d087ba/roles/be88bd87b07244debee4e9545284ff13 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:54:35.279884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e388b-0afa-4882-8255-f4bcbea9f384 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:35.280268 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e388b-0afa-4882-8255-f4bcbea9f384 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:35.280637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e388b-0afa-4882-8255-f4bcbea9f384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:35.310142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-042e388b-0afa-4882-8255-f4bcbea9f384 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:35.312738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-042e388b-0afa-4882-8255-f4bcbea9f384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:35.428330 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 404/814] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:35.429339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e388b-0afa-4882-8255-f4bcbea9f384 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:35.436422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becc1ef-1f57-4605-81e5-b33604fb3cd6 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:35.487273 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140322a-04db-4904-b380-74b32c02111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:35.488033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140322a-04db-4904-b380-74b32c02111b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:35.488325 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140322a-04db-4904-b380-74b32c02111b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:35.489166 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140322a-04db-4904-b380-74b32c02111b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:35.496837 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140322a-04db-4904-b380-74b32c02111b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:35.498362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140322a-04db-4904-b380-74b32c02111b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a140322a-04db-4904-b380-74b32c02111b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:35.515989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:35.516965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:35.517858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:35.517858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:35.517858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:35.517858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:35.517858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:35.517858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:35.517858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:35.517858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:35.520467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 411/815] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:35.521538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140322a-04db-4904-b380-74b32c02111b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:35.562893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:35.567317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e388b-0afa-4882-8255-f4bcbea9f384 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:35.595763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:35.604951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91112a72-1d54-4fcc-a1db-011becaee7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:35.605328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91112a72-1d54-4fcc-a1db-011becaee7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:35.605617 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91112a72-1d54-4fcc-a1db-011becaee7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:35.605897 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91112a72-1d54-4fcc-a1db-011becaee7a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:35.606728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91112a72-1d54-4fcc-a1db-011becaee7a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:35.607041 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91112a72-1d54-4fcc-a1db-011becaee7a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:35.608546 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91112a72-1d54-4fcc-a1db-011becaee7a4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:35.619119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 405/816] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:35.624606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91112a72-1d54-4fcc-a1db-011becaee7a4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:35.636267 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140322a-04db-4904-b380-74b32c02111b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:35.679889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0d9a2-dbd8-4fe6-882b-209557139db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:35.681714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0d9a2-dbd8-4fe6-882b-209557139db3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:35.682218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0d9a2-dbd8-4fe6-882b-209557139db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:35.682702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0d9a2-dbd8-4fe6-882b-209557139db3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:35.683888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0d9a2-dbd8-4fe6-882b-209557139db3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:35.685725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0d9a2-dbd8-4fe6-882b-209557139db3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b8b0d9a2-dbd8-4fe6-882b-209557139db3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:35.704779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:35.705845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:54:35.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:35.709787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 412/817] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:35 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:35.715695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0d9a2-dbd8-4fe6-882b-209557139db3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:35.733352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91112a72-1d54-4fcc-a1db-011becaee7a4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:35.769919 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15a862-191b-4011-9599-b99efd586da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:35.770643 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15a862-191b-4011-9599-b99efd586da3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:35.770841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15a862-191b-4011-9599-b99efd586da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:35.771122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15a862-191b-4011-9599-b99efd586da3 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:35.771962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15a862-191b-4011-9599-b99efd586da3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:35.772274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15a862-191b-4011-9599-b99efd586da3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:35.773746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15a862-191b-4011-9599-b99efd586da3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:35.792715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 406/818] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:35.793988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15a862-191b-4011-9599-b99efd586da3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:35.831296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0d9a2-dbd8-4fe6-882b-209557139db3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:35.838700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:35.848958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:35.873917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404d8c8-7366-41d0-a284-b588963ac5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:35.875134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404d8c8-7366-41d0-a284-b588963ac5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:35.875665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404d8c8-7366-41d0-a284-b588963ac5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:35.876156 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404d8c8-7366-41d0-a284-b588963ac5e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:35.888373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404d8c8-7366-41d0-a284-b588963ac5e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:35.890200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404d8c8-7366-41d0-a284-b588963ac5e9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:35.906873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15a862-191b-4011-9599-b99efd586da3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6404d8c8-7366-41d0-a284-b588963ac5e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:35.909277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:35.910147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:35.911608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:35.911608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:35.911608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:35.911608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:35.913957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 413/819] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:35.914980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404d8c8-7366-41d0-a284-b588963ac5e9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:35.951269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92a551-c9e2-4180-8067-916209bbaaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:35.952260 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92a551-c9e2-4180-8067-916209bbaaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:35.952559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92a551-c9e2-4180-8067-916209bbaaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:35.952903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92a551-c9e2-4180-8067-916209bbaaa0 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:35.953630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92a551-c9e2-4180-8067-916209bbaaa0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:35.953988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92a551-c9e2-4180-8067-916209bbaaa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:35.955702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92a551-c9e2-4180-8067-916209bbaaa0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:35.966784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 407/820] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:54:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:35.966784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92a551-c9e2-4180-8067-916209bbaaa0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:36.056056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404d8c8-7366-41d0-a284-b588963ac5e9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:36.069519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:36.087564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92a551-c9e2-4180-8067-916209bbaaa0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:36.101777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10781de0-9387-493b-bb9a-c1ad1ec42d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:36.102891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10781de0-9387-493b-bb9a-c1ad1ec42d84 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:36.103262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10781de0-9387-493b-bb9a-c1ad1ec42d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:36.103585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10781de0-9387-493b-bb9a-c1ad1ec42d84 None admin] PATH_INFO: `/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/978e71dfbb134f41871114824b03fbeb/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:36.106419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10781de0-9387-493b-bb9a-c1ad1ec42d84 None admin] RBAC: Authorizing `identity:create_grant(project_id=a68601f16d1c49548b6e3af0b9819ba3, user_id=978e71dfbb134f41871114824b03fbeb, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:36.108111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10781de0-9387-493b-bb9a-c1ad1ec42d84 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:36.128699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 414/821] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:54:35 2022] PUT /identity/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/978e71dfbb134f41871114824b03fbeb/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:54:36.129999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10781de0-9387-493b-bb9a-c1ad1ec42d84 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:36.146546 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4977e-3c5d-4618-8367-e5133cedcda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:36.150869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4977e-3c5d-4618-8367-e5133cedcda4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:36.151823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4977e-3c5d-4618-8367-e5133cedcda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:36.152576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4977e-3c5d-4618-8367-e5133cedcda4 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:36.155293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4977e-3c5d-4618-8367-e5133cedcda4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:36.157583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4977e-3c5d-4618-8367-e5133cedcda4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:36.159931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4977e-3c5d-4618-8367-e5133cedcda4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:36.184903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 408/822] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:36.192415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4977e-3c5d-4618-8367-e5133cedcda4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:36.196106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:36.379751 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10781de0-9387-493b-bb9a-c1ad1ec42d84 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:36.389555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4977e-3c5d-4618-8367-e5133cedcda4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:36.454026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6eb265-6685-42c1-b556-aa077fa7eba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:36.454718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eb265-6685-42c1-b556-aa077fa7eba5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:36.455004 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eb265-6685-42c1-b556-aa077fa7eba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:36.455292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eb265-6685-42c1-b556-aa077fa7eba5 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:36.456076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6eb265-6685-42c1-b556-aa077fa7eba5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:36.456539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6eb265-6685-42c1-b556-aa077fa7eba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:36.459488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6eb265-6685-42c1-b556-aa077fa7eba5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:36.463617 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7256d0e-e1eb-454e-ab10-1bd7bce7a3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:36.464460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7256d0e-e1eb-454e-ab10-1bd7bce7a3ef None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:36.464742 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7256d0e-e1eb-454e-ab10-1bd7bce7a3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:36.465021 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7256d0e-e1eb-454e-ab10-1bd7bce7a3ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:36.476336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7256d0e-e1eb-454e-ab10-1bd7bce7a3ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:36.478029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 415/823] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:54:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:36.478155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7256d0e-e1eb-454e-ab10-1bd7bce7a3ef None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:36.484181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6eb265-6685-42c1-b556-aa077fa7eba5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f7256d0e-e1eb-454e-ab10-1bd7bce7a3ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:36.512600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:36.513606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:36.514542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:36.514542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:36.514542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:36.514542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:36.514542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:36.514542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:36.514542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:36.514542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:36.514542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:54:36.514542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:36.514542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 409/824] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Jul 14 10:54:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:36.523795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7256d0e-e1eb-454e-ab10-1bd7bce7a3ef None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:36.544441 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:36.662585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:36.681145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6eb265-6685-42c1-b556-aa077fa7eba5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:36.736251 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7256d0e-e1eb-454e-ab10-1bd7bce7a3ef None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:36.753279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176102bc-797f-45ca-9407-245ee72d8e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:36.754885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176102bc-797f-45ca-9407-245ee72d8e43 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:36.755727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176102bc-797f-45ca-9407-245ee72d8e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:36.757668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176102bc-797f-45ca-9407-245ee72d8e43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:36.758703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176102bc-797f-45ca-9407-245ee72d8e43 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:36.760354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176102bc-797f-45ca-9407-245ee72d8e43 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:36.797231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 416/825] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Jul 14 10:54:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:54:36.815015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa01a1-a259-44e8-b747-9f20efe3d42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:36.816198 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa01a1-a259-44e8-b747-9f20efe3d42e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:36.817313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa01a1-a259-44e8-b747-9f20efe3d42e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:36.817892 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa01a1-a259-44e8-b747-9f20efe3d42e None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:36.819000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa01a1-a259-44e8-b747-9f20efe3d42e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:36.819598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa01a1-a259-44e8-b747-9f20efe3d42e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:36.821508 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa01a1-a259-44e8-b747-9f20efe3d42e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:36.843302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 410/826] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Jul 14 10:54:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:36.849675 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176102bc-797f-45ca-9407-245ee72d8e43 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:36.952663 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ff321-74d1-4565-a36c-ae0fd16f33a2 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:36.952860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ff321-74d1-4565-a36c-ae0fd16f33a2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:36.953146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ff321-74d1-4565-a36c-ae0fd16f33a2 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:36.954809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 411/827] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:36.960454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830233ed-ced3-4ff6-9677-ba2d004bdfb3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:36.960777 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830233ed-ced3-4ff6-9677-ba2d004bdfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:36.961019 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830233ed-ced3-4ff6-9677-ba2d004bdfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:36.986240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176102bc-797f-45ca-9407-245ee72d8e43 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:36.997941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-830233ed-ced3-4ff6-9677-ba2d004bdfb3 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:36.999184 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-830233ed-ced3-4ff6-9677-ba2d004bdfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:37.055510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a714f39-b848-4347-b32e-661723326d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:37.056191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a714f39-b848-4347-b32e-661723326d3b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:37.056474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a714f39-b848-4347-b32e-661723326d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:37.056751 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a714f39-b848-4347-b32e-661723326d3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:37.101908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:37.132366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 412/828] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:54:37.132927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:37.171896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830233ed-ced3-4ff6-9677-ba2d004bdfb3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:37.179934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a714f39-b848-4347-b32e-661723326d3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:37.185297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a714f39-b848-4347-b32e-661723326d3b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:37.311771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830233ed-ced3-4ff6-9677-ba2d004bdfb3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:37.324061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 417/829] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:54:37.335423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a714f39-b848-4347-b32e-661723326d3b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:37.362219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f8146-d47e-4e6f-ac69-b3a746de5dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:37.363142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f8146-d47e-4e6f-ac69-b3a746de5dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:37.364234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f8146-d47e-4e6f-ac69-b3a746de5dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:37.364521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f8146-d47e-4e6f-ac69-b3a746de5dc3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:37.365416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f8146-d47e-4e6f-ac69-b3a746de5dc3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:37.367837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f8146-d47e-4e6f-ac69-b3a746de5dc3 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-2e1f8146-d47e-4e6f-ac69-b3a746de5dc3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:37.387160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:37.388239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:37.389217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 413/830] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:54:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:37.397656 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f8146-d47e-4e6f-ac69-b3a746de5dc3 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:37.400900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:37.458894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:37.468043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a714f39-b848-4347-b32e-661723326d3b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:37.507478 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542c56e-f826-4fbd-9e03-f50ce972d511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:37.508159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f8146-d47e-4e6f-ac69-b3a746de5dc3 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:37.508890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542c56e-f826-4fbd-9e03-f50ce972d511 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:37.509375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542c56e-f826-4fbd-9e03-f50ce972d511 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:37.509985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542c56e-f826-4fbd-9e03-f50ce972d511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:37.524416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542c56e-f826-4fbd-9e03-f50ce972d511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:37.527557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542c56e-f826-4fbd-9e03-f50ce972d511 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:37.546501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ef425-4f12-4a56-b987-6963f056f683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:37.547169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ef425-4f12-4a56-b987-6963f056f683 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:37.547473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ef425-4f12-4a56-b987-6963f056f683 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:37.547753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ef425-4f12-4a56-b987-6963f056f683 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:37.548662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ef425-4f12-4a56-b987-6963f056f683 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:37.549085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ef425-4f12-4a56-b987-6963f056f683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:37.550478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ef425-4f12-4a56-b987-6963f056f683 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-f542c56e-f826-4fbd-9e03-f50ce972d511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:37.552446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:37.553301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:37.554201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:37.554201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:37.554201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:37.554201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:37.556512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 418/831] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:37.561594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 414/832] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:54:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:37.562614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ef425-4f12-4a56-b987-6963f056f683 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:37.562838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542c56e-f826-4fbd-9e03-f50ce972d511 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:37.680509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ef425-4f12-4a56-b987-6963f056f683 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:37.684751 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:37.688409 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542c56e-f826-4fbd-9e03-f50ce972d511 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:37.702121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:37.724455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036942d4-6c15-48f4-ad87-bc77564d5dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:37.725239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036942d4-6c15-48f4-ad87-bc77564d5dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:37.725599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036942d4-6c15-48f4-ad87-bc77564d5dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:37.725955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036942d4-6c15-48f4-ad87-bc77564d5dc5 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:37.726742 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036942d4-6c15-48f4-ad87-bc77564d5dc5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:37.727122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036942d4-6c15-48f4-ad87-bc77564d5dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:37.728181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c37da-3d4c-4b3e-85ec-2d4ace8ca2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:37.728640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036942d4-6c15-48f4-ad87-bc77564d5dc5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:37.728856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c37da-3d4c-4b3e-85ec-2d4ace8ca2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:37.729141 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c37da-3d4c-4b3e-85ec-2d4ace8ca2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:37.729419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c37da-3d4c-4b3e-85ec-2d4ace8ca2a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:37.730150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c37da-3d4c-4b3e-85ec-2d4ace8ca2a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:37.730451 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c37da-3d4c-4b3e-85ec-2d4ace8ca2a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:37.731855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c37da-3d4c-4b3e-85ec-2d4ace8ca2a8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:37.740212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 415/833] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:54:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:37.740212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7bf50-ddf8-427f-9435-df4dbe6ffb5c None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:37.740212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7bf50-ddf8-427f-9435-df4dbe6ffb5c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:37.740212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7bf50-ddf8-427f-9435-df4dbe6ffb5c None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:37.740212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 416/834] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:37.744976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d61d09-1432-489b-88ca-c9b92d12e493 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:37.745266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d61d09-1432-489b-88ca-c9b92d12e493 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:37.745554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d61d09-1432-489b-88ca-c9b92d12e493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:37.756140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 419/835] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:37.766797 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c37da-3d4c-4b3e-85ec-2d4ace8ca2a8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:37.773133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-10d61d09-1432-489b-88ca-c9b92d12e493 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:37.774637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-10d61d09-1432-489b-88ca-c9b92d12e493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:37.876038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 417/836] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:37.900226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d61d09-1432-489b-88ca-c9b92d12e493 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:37.906938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c37da-3d4c-4b3e-85ec-2d4ace8ca2a8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:37.963299 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b0b8d-2769-40a3-a3f3-fe61a96ce41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:37.964532 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b0b8d-2769-40a3-a3f3-fe61a96ce41c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:37.964890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b0b8d-2769-40a3-a3f3-fe61a96ce41c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:37.965242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b0b8d-2769-40a3-a3f3-fe61a96ce41c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:37.974230 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b0b8d-2769-40a3-a3f3-fe61a96ce41c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:37.976060 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b0b8d-2769-40a3-a3f3-fe61a96ce41c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a00b0b8d-2769-40a3-a3f3-fe61a96ce41c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:37.991567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:37.992563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:37.993446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:37.993446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:37.993446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:37.993446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:37.993446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:37.993446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:37.993446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:37.993446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:37.995705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 420/837] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:38.006967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b0b8d-2769-40a3-a3f3-fe61a96ce41c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:38.032848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:38.048767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d61d09-1432-489b-88ca-c9b92d12e493 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:38.068161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:38.096810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56655d48-8f5e-4512-9167-5ff8e3169f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:38.098359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56655d48-8f5e-4512-9167-5ff8e3169f1b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:38.098872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56655d48-8f5e-4512-9167-5ff8e3169f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:38.099356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56655d48-8f5e-4512-9167-5ff8e3169f1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:38.115706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56655d48-8f5e-4512-9167-5ff8e3169f1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-56655d48-8f5e-4512-9167-5ff8e3169f1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:38.128043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:38.129017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:38.130403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:38.130403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:38.130403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:38.130403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:38.130403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:38.130403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:38.130403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:38.130403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:38.130403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:38.130403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:38.130403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:54:38.130403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:38.138672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 418/838] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:54:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:38.140771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56655d48-8f5e-4512-9167-5ff8e3169f1b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:38.152549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b0b8d-2769-40a3-a3f3-fe61a96ce41c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:38.198035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a78ce-2b57-46a2-965c-c8545fb759ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:38.199783 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a78ce-2b57-46a2-965c-c8545fb759ca None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:38.200268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a78ce-2b57-46a2-965c-c8545fb759ca None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:38.200742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a78ce-2b57-46a2-965c-c8545fb759ca None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:38.202270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a78ce-2b57-46a2-965c-c8545fb759ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:38.202797 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a78ce-2b57-46a2-965c-c8545fb759ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:38.205060 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a78ce-2b57-46a2-965c-c8545fb759ca None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:38.225164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 421/839] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:38.234735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a78ce-2b57-46a2-965c-c8545fb759ca None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:38.277993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56655d48-8f5e-4512-9167-5ff8e3169f1b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:38.323920 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcefd8-a40e-4344-8579-dfbfcd8c5206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:38.324899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcefd8-a40e-4344-8579-dfbfcd8c5206 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:38.325191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcefd8-a40e-4344-8579-dfbfcd8c5206 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:38.325495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcefd8-a40e-4344-8579-dfbfcd8c5206 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:38.327287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcefd8-a40e-4344-8579-dfbfcd8c5206 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:38.327627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcefd8-a40e-4344-8579-dfbfcd8c5206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:38.329135 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcefd8-a40e-4344-8579-dfbfcd8c5206 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:38.336906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:38.343821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 419/840] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:54:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:38.349287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcefd8-a40e-4344-8579-dfbfcd8c5206 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:38.367108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a78ce-2b57-46a2-965c-c8545fb759ca None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:38.397415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:38.409713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96bd0a-c276-4b41-bef3-48cff9a20c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:38.410837 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96bd0a-c276-4b41-bef3-48cff9a20c60 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:38.411352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96bd0a-c276-4b41-bef3-48cff9a20c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:38.411867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96bd0a-c276-4b41-bef3-48cff9a20c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:38.412911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96bd0a-c276-4b41-bef3-48cff9a20c60 None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=7e572bf785284a8693cf386ced004524) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:38.413457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96bd0a-c276-4b41-bef3-48cff9a20c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:38.415235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96bd0a-c276-4b41-bef3-48cff9a20c60 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:38.427843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 422/841] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:38 2022] GET /identity/v3/role_assignments?user.id=7e572bf785284a8693cf386ced004524&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:38.481179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcefd8-a40e-4344-8579-dfbfcd8c5206 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:38.482720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640cd92-9981-4aa9-80db-82e3554f51e9 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:38.483193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640cd92-9981-4aa9-80db-82e3554f51e9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:38.483413 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640cd92-9981-4aa9-80db-82e3554f51e9 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:38.485128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 423/842] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:38.492549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5971c8-af6d-4b68-88f6-24257f466b9d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:38.492866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5971c8-af6d-4b68-88f6-24257f466b9d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:38.493583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5971c8-af6d-4b68-88f6-24257f466b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:38.516784 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5d5971c8-af6d-4b68-88f6-24257f466b9d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:38.518763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5d5971c8-af6d-4b68-88f6-24257f466b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:38.522771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e7842-ca51-4dae-a089-dc5d23429309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:38.523664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e7842-ca51-4dae-a089-dc5d23429309 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:38.523952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e7842-ca51-4dae-a089-dc5d23429309 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:38.524238 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e7842-ca51-4dae-a089-dc5d23429309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:38.538128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e7842-ca51-4dae-a089-dc5d23429309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:38.539871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e7842-ca51-4dae-a089-dc5d23429309 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-532e7842-ca51-4dae-a089-dc5d23429309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:38.564123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:38.565063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:38.565950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:38.565950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:38.565950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:38.565950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:38.570445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 420/843] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:38.575416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e7842-ca51-4dae-a089-dc5d23429309 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:38.618956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 424/844] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:38.645735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5971c8-af6d-4b68-88f6-24257f466b9d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:38.686399 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:38.710407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e7842-ca51-4dae-a089-dc5d23429309 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:38.753846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3306c3-b30e-46d7-8b0d-2c1002e95759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:38.754542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:38.754680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3306c3-b30e-46d7-8b0d-2c1002e95759 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:38.754860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3306c3-b30e-46d7-8b0d-2c1002e95759 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:38.755106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3306c3-b30e-46d7-8b0d-2c1002e95759 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:38.755871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3306c3-b30e-46d7-8b0d-2c1002e95759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:38.756987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3306c3-b30e-46d7-8b0d-2c1002e95759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:38.757847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3306c3-b30e-46d7-8b0d-2c1002e95759 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:38.771861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 421/845] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:38.776946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3306c3-b30e-46d7-8b0d-2c1002e95759 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:38.801892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5971c8-af6d-4b68-88f6-24257f466b9d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:38.840195 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0460508-2a87-47fa-87e3-92e86e2e254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:38.841283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0460508-2a87-47fa-87e3-92e86e2e254a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:38.841713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0460508-2a87-47fa-87e3-92e86e2e254a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:38.842171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0460508-2a87-47fa-87e3-92e86e2e254a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:38.854587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0460508-2a87-47fa-87e3-92e86e2e254a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-f0460508-2a87-47fa-87e3-92e86e2e254a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:38.863140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:38.864950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:38.865785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:38.865785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:38.865785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:38.865785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:38.865785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:38.865785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:38.865785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:38.865785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:38.865785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:38.865785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:38.865785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:54:38.865785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:38.872771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 425/846] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:54:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:38.873968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0460508-2a87-47fa-87e3-92e86e2e254a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:38.891591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3306c3-b30e-46d7-8b0d-2c1002e95759 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:38.930643 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cbacd-0d9c-4369-b8e5-649ed9793d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:38.931586 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cbacd-0d9c-4369-b8e5-649ed9793d61 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:38.931870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cbacd-0d9c-4369-b8e5-649ed9793d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:38.932142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cbacd-0d9c-4369-b8e5-649ed9793d61 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:38.940652 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cbacd-0d9c-4369-b8e5-649ed9793d61 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:38.942189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cbacd-0d9c-4369-b8e5-649ed9793d61 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-7b3cbacd-0d9c-4369-b8e5-649ed9793d61 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:38.959043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:38.960302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:38.961418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:38.961418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:38.961418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:38.961418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:38.961418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:38.961418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:38.961418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:38.961418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:38.961418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:54:38.961418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:38.965556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 422/847] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Jul 14 10:54:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:38.966577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cbacd-0d9c-4369-b8e5-649ed9793d61 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:38.976996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:38.997963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0460508-2a87-47fa-87e3-92e86e2e254a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:39.038126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:39.048135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf500bd-4e0f-4054-a4c2-1dfd0d1e2a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:39.051975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf500bd-4e0f-4054-a4c2-1dfd0d1e2a5f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:39.052423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf500bd-4e0f-4054-a4c2-1dfd0d1e2a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:39.052862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf500bd-4e0f-4054-a4c2-1dfd0d1e2a5f None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:39.053640 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf500bd-4e0f-4054-a4c2-1dfd0d1e2a5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:39.053952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf500bd-4e0f-4054-a4c2-1dfd0d1e2a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:39.055346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf500bd-4e0f-4054-a4c2-1dfd0d1e2a5f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:39.070899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 426/848] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:39.072562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf500bd-4e0f-4054-a4c2-1dfd0d1e2a5f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:39.095621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cbacd-0d9c-4369-b8e5-649ed9793d61 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:39.137844 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c07258-207c-4663-9a91-63b0d82cbc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:39.138523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c07258-207c-4663-9a91-63b0d82cbc59 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:39.138803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c07258-207c-4663-9a91-63b0d82cbc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:39.139078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c07258-207c-4663-9a91-63b0d82cbc59 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:39.139953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c07258-207c-4663-9a91-63b0d82cbc59 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:39.140263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c07258-207c-4663-9a91-63b0d82cbc59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:39.142325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c07258-207c-4663-9a91-63b0d82cbc59 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:39.169623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 423/849] 10.222.0.75 () {56 vars in 1149 bytes} [Thu Jul 14 10:54:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:39.176244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c07258-207c-4663-9a91-63b0d82cbc59 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:39.219249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf500bd-4e0f-4054-a4c2-1dfd0d1e2a5f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:39.256008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:39.258567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b19bd-88d6-4aec-aae3-446be5292b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:39.259317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b19bd-88d6-4aec-aae3-446be5292b8d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:39.259632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b19bd-88d6-4aec-aae3-446be5292b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:39.259909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b19bd-88d6-4aec-aae3-446be5292b8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:39.275937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b19bd-88d6-4aec-aae3-446be5292b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:39.277407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b19bd-88d6-4aec-aae3-446be5292b8d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a85b19bd-88d6-4aec-aae3-446be5292b8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:39.296538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:39.297522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:39.298381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:39.298381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:39.298381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:39.298381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:39.302659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 427/850] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:39.303568 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b19bd-88d6-4aec-aae3-446be5292b8d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:39.313106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c07258-207c-4663-9a91-63b0d82cbc59 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:39.355971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:39.368278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65f01e-a855-437e-89a2-fdb043c580a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:39.370239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65f01e-a855-437e-89a2-fdb043c580a0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:39.370672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65f01e-a855-437e-89a2-fdb043c580a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:39.371136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65f01e-a855-437e-89a2-fdb043c580a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:39.431530 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b19bd-88d6-4aec-aae3-446be5292b8d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:39.456869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65f01e-a855-437e-89a2-fdb043c580a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:39.459004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65f01e-a855-437e-89a2-fdb043c580a0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:39.480490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96d69d-8afc-4884-a513-c32e295bd989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:39.481987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96d69d-8afc-4884-a513-c32e295bd989 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:39.482422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96d69d-8afc-4884-a513-c32e295bd989 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:39.483199 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96d69d-8afc-4884-a513-c32e295bd989 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:39.484820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96d69d-8afc-4884-a513-c32e295bd989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:39.485733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96d69d-8afc-4884-a513-c32e295bd989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:39.487617 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96d69d-8afc-4884-a513-c32e295bd989 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:39.505835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 428/851] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:39.511130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96d69d-8afc-4884-a513-c32e295bd989 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:39.532749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:39.593867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 424/852] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:54:39.595158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65f01e-a855-437e-89a2-fdb043c580a0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:39.641871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96d69d-8afc-4884-a513-c32e295bd989 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:39.652690 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:39.695001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71e0c5-8596-4de5-9d13-46d99e8a57a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:39.696218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71e0c5-8596-4de5-9d13-46d99e8a57a1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:39.696540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71e0c5-8596-4de5-9d13-46d99e8a57a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:39.696830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71e0c5-8596-4de5-9d13-46d99e8a57a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:39.706368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71e0c5-8596-4de5-9d13-46d99e8a57a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:39.709028 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71e0c5-8596-4de5-9d13-46d99e8a57a1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:39.717523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65f01e-a855-437e-89a2-fdb043c580a0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-5d71e0c5-8596-4de5-9d13-46d99e8a57a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:39.727983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:39.728972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:39.729906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:39.729906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:39.729906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:39.729906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:39.729906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:39.729906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:39.729906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:39.729906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:39.729906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:54:39.729906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:39.731464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 429/853] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:54:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:39.735138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71e0c5-8596-4de5-9d13-46d99e8a57a1 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:39.784168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec85e6-b231-42b4-bed2-07b0ea6de46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:39.786008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec85e6-b231-42b4-bed2-07b0ea6de46b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:39.786505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec85e6-b231-42b4-bed2-07b0ea6de46b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:39.787478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec85e6-b231-42b4-bed2-07b0ea6de46b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:39.800282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec85e6-b231-42b4-bed2-07b0ea6de46b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:39.801777 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec85e6-b231-42b4-bed2-07b0ea6de46b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-d6ec85e6-b231-42b4-bed2-07b0ea6de46b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:39.821307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:39.822278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:39.823177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:39.823177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:39.823177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:39.823177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:39.827417 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 425/854] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:39.828495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec85e6-b231-42b4-bed2-07b0ea6de46b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:39.849354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:39.873275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71e0c5-8596-4de5-9d13-46d99e8a57a1 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:39.913979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8256b-2780-42df-a28c-e7df1e37f024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:39.915293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8256b-2780-42df-a28c-e7df1e37f024 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:39.915806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8256b-2780-42df-a28c-e7df1e37f024 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:39.916284 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8256b-2780-42df-a28c-e7df1e37f024 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:39.917371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8256b-2780-42df-a28c-e7df1e37f024 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:39.917884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8256b-2780-42df-a28c-e7df1e37f024 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:39.920127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8256b-2780-42df-a28c-e7df1e37f024 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:39.940758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 430/855] 10.222.0.75 () {56 vars in 1135 bytes} [Thu Jul 14 10:54:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:39.942580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ee9aa-ebdc-4735-9e1f-15d7b3068cde None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:39.942857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ee9aa-ebdc-4735-9e1f-15d7b3068cde None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:39.943132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ee9aa-ebdc-4735-9e1f-15d7b3068cde None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:39.944596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 431/856] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:39.948587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ee9aa-ebdc-4735-9e1f-15d7b3068cde None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:39.964821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec85e6-b231-42b4-bed2-07b0ea6de46b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:39.988905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:40.014785 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6885e-28db-4e0f-aac3-a38e8d70594a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:40.015756 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6885e-28db-4e0f-aac3-a38e8d70594a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:40.016227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6885e-28db-4e0f-aac3-a38e8d70594a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:40.016659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6885e-28db-4e0f-aac3-a38e8d70594a None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:40.017559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6885e-28db-4e0f-aac3-a38e8d70594a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:40.018035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6885e-28db-4e0f-aac3-a38e8d70594a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:40.019451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6885e-28db-4e0f-aac3-a38e8d70594a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:40.035895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 426/857] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:40.035895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd127fce-8d17-4c22-b6b3-46bddae19fa2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:40.035895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd127fce-8d17-4c22-b6b3-46bddae19fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:40.035895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd127fce-8d17-4c22-b6b3-46bddae19fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:40.060093 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-dd127fce-8d17-4c22-b6b3-46bddae19fa2 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:40.061306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-dd127fce-8d17-4c22-b6b3-46bddae19fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:40.082596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ee9aa-ebdc-4735-9e1f-15d7b3068cde None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:40.121461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b7557-fc26-474e-bae2-609dd262402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:40.122285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b7557-fc26-474e-bae2-609dd262402b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:40.122680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b7557-fc26-474e-bae2-609dd262402b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:40.122973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b7557-fc26-474e-bae2-609dd262402b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:40.142219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 427/858] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:40.143599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd127fce-8d17-4c22-b6b3-46bddae19fa2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:40.167051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:40.225045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b7557-fc26-474e-bae2-609dd262402b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:40.226413 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b7557-fc26-474e-bae2-609dd262402b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:40.262144 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:40.297083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd127fce-8d17-4c22-b6b3-46bddae19fa2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:40.334911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb9c7c-01de-4b05-9d30-b76baae6c92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:40.335743 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb9c7c-01de-4b05-9d30-b76baae6c92a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:40.336029 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb9c7c-01de-4b05-9d30-b76baae6c92a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:40.336307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb9c7c-01de-4b05-9d30-b76baae6c92a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:40.343866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb9c7c-01de-4b05-9d30-b76baae6c92a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:40.345387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb9c7c-01de-4b05-9d30-b76baae6c92a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:40.361007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 432/859] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:54:40.361007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b7557-fc26-474e-bae2-609dd262402b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-75bb9c7c-01de-4b05-9d30-b76baae6c92a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:40.361299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:40.362091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:40.362979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:40.362979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:40.362979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:40.362979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:40.362979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:40.362979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:40.362979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:40.362979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:40.362979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 428/860] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:40.362979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb9c7c-01de-4b05-9d30-b76baae6c92a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:40.453700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:40.484612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b7557-fc26-474e-bae2-609dd262402b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:40.504265 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb9c7c-01de-4b05-9d30-b76baae6c92a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:40.517055 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:40.536419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34e304-884e-42a2-93b6-610a0970512c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:40.537135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34e304-884e-42a2-93b6-610a0970512c None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:40.537391 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34e304-884e-42a2-93b6-610a0970512c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:40.537772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34e304-884e-42a2-93b6-610a0970512c None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:40.538408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34e304-884e-42a2-93b6-610a0970512c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:40.539832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34e304-884e-42a2-93b6-610a0970512c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:40.542290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820f47b-7430-4d87-aa82-a722a8251419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:40.543110 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820f47b-7430-4d87-aa82-a722a8251419 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:40.543449 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820f47b-7430-4d87-aa82-a722a8251419 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:40.543742 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820f47b-7430-4d87-aa82-a722a8251419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:40.556476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820f47b-7430-4d87-aa82-a722a8251419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:40.558206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820f47b-7430-4d87-aa82-a722a8251419 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:40.563565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 433/861] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:54:40 2022] POST /identity/v3/services => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:54:40.563565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34e304-884e-42a2-93b6-610a0970512c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-9820f47b-7430-4d87-aa82-a722a8251419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:40.586985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:40.588056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:40.588976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:40.588976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:40.588976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:40.588976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:40.588976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 429/862] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:40.595905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820f47b-7430-4d87-aa82-a722a8251419 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:40.711543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34e304-884e-42a2-93b6-610a0970512c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:40.737436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820f47b-7430-4d87-aa82-a722a8251419 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:40.762447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eab73c-b467-4d8f-a846-1e19a9a0b54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:40.763136 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eab73c-b467-4d8f-a846-1e19a9a0b54d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:40.763496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eab73c-b467-4d8f-a846-1e19a9a0b54d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:40.763784 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eab73c-b467-4d8f-a846-1e19a9a0b54d None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:40.764073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:40.764585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eab73c-b467-4d8f-a846-1e19a9a0b54d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:40.764892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eab73c-b467-4d8f-a846-1e19a9a0b54d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:40.766353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eab73c-b467-4d8f-a846-1e19a9a0b54d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:40.781425 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 434/863] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:40.782784 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88e7f9-b99a-40b8-aff1-d1557a5e9275 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:40.783082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88e7f9-b99a-40b8-aff1-d1557a5e9275 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:40.783368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88e7f9-b99a-40b8-aff1-d1557a5e9275 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:40.787323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 435/864] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:40.790343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f198282-0b45-4638-a379-af90989a9b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:40.790343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f198282-0b45-4638-a379-af90989a9b56 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:40.790609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88e7f9-b99a-40b8-aff1-d1557a5e9275 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:40.790696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f198282-0b45-4638-a379-af90989a9b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:40.790696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f198282-0b45-4638-a379-af90989a9b56 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:40.790696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f198282-0b45-4638-a379-af90989a9b56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:40.790696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f198282-0b45-4638-a379-af90989a9b56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:40.791761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f198282-0b45-4638-a379-af90989a9b56 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:40.804883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 430/865] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:40.806208 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5013b7b-2a36-4277-bc1c-d22e5b4aa431 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:40.806496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5013b7b-2a36-4277-bc1c-d22e5b4aa431 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:40.806841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5013b7b-2a36-4277-bc1c-d22e5b4aa431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:40.825792 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d5013b7b-2a36-4277-bc1c-d22e5b4aa431 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:40.827004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d5013b7b-2a36-4277-bc1c-d22e5b4aa431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:40.855632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:40.913581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 431/866] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:40 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:40.923611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5013b7b-2a36-4277-bc1c-d22e5b4aa431 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:40.948507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88e7f9-b99a-40b8-aff1-d1557a5e9275 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:40.999647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6323cd-b59b-4be3-90e7-26854e77528a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:41.000237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6323cd-b59b-4be3-90e7-26854e77528a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:41.000530 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6323cd-b59b-4be3-90e7-26854e77528a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:41.000807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6323cd-b59b-4be3-90e7-26854e77528a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:41.001645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6323cd-b59b-4be3-90e7-26854e77528a None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=f40c0fcc93cf4bbf8d35c1de70d087ba) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:41.001961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6323cd-b59b-4be3-90e7-26854e77528a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:41.003435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6323cd-b59b-4be3-90e7-26854e77528a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:41.010855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 436/867] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:40 2022] GET /identity/v3/role_assignments?user.id=f40c0fcc93cf4bbf8d35c1de70d087ba&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:41.011648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6323cd-b59b-4be3-90e7-26854e77528a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:41.042034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5013b7b-2a36-4277-bc1c-d22e5b4aa431 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:41.057384 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:41.100871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563af1c-3689-423e-8699-325c05552b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:41.101704 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563af1c-3689-423e-8699-325c05552b60 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:41.101999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563af1c-3689-423e-8699-325c05552b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:41.102273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563af1c-3689-423e-8699-325c05552b60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:41.118807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563af1c-3689-423e-8699-325c05552b60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:41.121534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563af1c-3689-423e-8699-325c05552b60 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-5563af1c-3689-423e-8699-325c05552b60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:41.146076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:41.147027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:41.147930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:41.147930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:41.147930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:41.147930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:41.147930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:41.147930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:41.147930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:41.147930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:41.151583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 432/868] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:41.155577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563af1c-3689-423e-8699-325c05552b60 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:41.159911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6323cd-b59b-4be3-90e7-26854e77528a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:41.170805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:41.228493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dabee-f343-4c71-bd10-5a85db02ca95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:41.229343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dabee-f343-4c71-bd10-5a85db02ca95 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:41.229637 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dabee-f343-4c71-bd10-5a85db02ca95 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:41.229900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dabee-f343-4c71-bd10-5a85db02ca95 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:41.230568 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9dabee-f343-4c71-bd10-5a85db02ca95 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:41.232012 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9dabee-f343-4c71-bd10-5a85db02ca95 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6d9dabee-f343-4c71-bd10-5a85db02ca95 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:41.248910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:41.250071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:41.251177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 437/869] 10.222.0.75 () {56 vars in 1132 bytes} [Thu Jul 14 10:54:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:41.256911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dabee-f343-4c71-bd10-5a85db02ca95 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:41.344853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563af1c-3689-423e-8699-325c05552b60 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:41.389686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bcab3-aa58-4d65-946c-c4e3d4e8ed0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:41.390364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bcab3-aa58-4d65-946c-c4e3d4e8ed0e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:41.390648 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bcab3-aa58-4d65-946c-c4e3d4e8ed0e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:41.390923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bcab3-aa58-4d65-946c-c4e3d4e8ed0e None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:41.391747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bcab3-aa58-4d65-946c-c4e3d4e8ed0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:41.392054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bcab3-aa58-4d65-946c-c4e3d4e8ed0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:41.393527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bcab3-aa58-4d65-946c-c4e3d4e8ed0e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:41.398755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dabee-f343-4c71-bd10-5a85db02ca95 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:41.411430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 433/870] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:41.415211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bcab3-aa58-4d65-946c-c4e3d4e8ed0e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:41.452482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33646ab2-b0d1-4ff4-beb2-94a932184d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:41.453221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33646ab2-b0d1-4ff4-beb2-94a932184d96 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:41.453507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33646ab2-b0d1-4ff4-beb2-94a932184d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:41.453789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33646ab2-b0d1-4ff4-beb2-94a932184d96 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:41.454556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33646ab2-b0d1-4ff4-beb2-94a932184d96 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:41.454872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33646ab2-b0d1-4ff4-beb2-94a932184d96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:41.456893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33646ab2-b0d1-4ff4-beb2-94a932184d96 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:41.461495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:41.467666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 438/871] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Jul 14 10:54:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:41.469893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33646ab2-b0d1-4ff4-beb2-94a932184d96 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:41.508695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:41.547673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bcab3-aa58-4d65-946c-c4e3d4e8ed0e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:41.586612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33646ab2-b0d1-4ff4-beb2-94a932184d96 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:41.587642 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede50c60-0945-4309-a85f-4c3990b01091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:41.588394 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede50c60-0945-4309-a85f-4c3990b01091 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:41.588745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede50c60-0945-4309-a85f-4c3990b01091 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:41.589092 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede50c60-0945-4309-a85f-4c3990b01091 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:41.589991 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede50c60-0945-4309-a85f-4c3990b01091 None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=978e71dfbb134f41871114824b03fbeb) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:41.590370 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede50c60-0945-4309-a85f-4c3990b01091 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:41.591919 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede50c60-0945-4309-a85f-4c3990b01091 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:41.599421 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 434/872] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:41 2022] GET /identity/v3/role_assignments?user.id=978e71dfbb134f41871114824b03fbeb&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:41.638397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf7084-6e03-4ba8-b0fe-75954082740e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:41.639098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf7084-6e03-4ba8-b0fe-75954082740e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:41.639425 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf7084-6e03-4ba8-b0fe-75954082740e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:41.639701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf7084-6e03-4ba8-b0fe-75954082740e None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:41.640436 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf7084-6e03-4ba8-b0fe-75954082740e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:41.640748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf7084-6e03-4ba8-b0fe-75954082740e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:41.642171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf7084-6e03-4ba8-b0fe-75954082740e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:41.655492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 439/873] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Jul 14 10:54:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:42.769031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dd44d-c30a-4632-a522-2af5772dc8eb None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:42.769547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dd44d-c30a-4632-a522-2af5772dc8eb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:42.769691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dd44d-c30a-4632-a522-2af5772dc8eb None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:42.771810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 435/874] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:42.777682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0aa47b-1689-40b0-b77e-871d38590a9d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:42.778236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0aa47b-1689-40b0-b77e-871d38590a9d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:42.778236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0aa47b-1689-40b0-b77e-871d38590a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:42.797337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ae0aa47b-1689-40b0-b77e-871d38590a9d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:42.798593 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ae0aa47b-1689-40b0-b77e-871d38590a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:42.882732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 440/875] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:42 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:54:42.905050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411dd44d-c30a-4632-a522-2af5772dc8eb None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:43.026416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:43.037797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411dd44d-c30a-4632-a522-2af5772dc8eb None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:43.082604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa64ea6-3387-40c4-83f6-6a4b488ae06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:43.083552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa64ea6-3387-40c4-83f6-6a4b488ae06d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.083843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa64ea6-3387-40c4-83f6-6a4b488ae06d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.084206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa64ea6-3387-40c4-83f6-6a4b488ae06d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.084983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa64ea6-3387-40c4-83f6-6a4b488ae06d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:43.086364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa64ea6-3387-40c4-83f6-6a4b488ae06d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-cfa64ea6-3387-40c4-83f6-6a4b488ae06d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:43.099419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:43.100514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:43.101871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 436/876] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:54:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:43.107874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0aa47b-1689-40b0-b77e-871d38590a9d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:43.135835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:43.217649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0aa47b-1689-40b0-b77e-871d38590a9d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:43.254577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d7fa2-a858-4d1b-9ed5-b9ac0469eb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:43.255264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d7fa2-a858-4d1b-9ed5-b9ac0469eb9c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.255597 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d7fa2-a858-4d1b-9ed5-b9ac0469eb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.255886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d7fa2-a858-4d1b-9ed5-b9ac0469eb9c None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.256608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d7fa2-a858-4d1b-9ed5-b9ac0469eb9c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:43.256978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d7fa2-a858-4d1b-9ed5-b9ac0469eb9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:43.258371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d7fa2-a858-4d1b-9ed5-b9ac0469eb9c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:43.263910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 441/877] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:54:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:43.269221 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa64ea6-3387-40c4-83f6-6a4b488ae06d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:43.295099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcc9b4-c688-4dab-a243-52cc9cb092d8 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.295485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcc9b4-c688-4dab-a243-52cc9cb092d8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.295620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcc9b4-c688-4dab-a243-52cc9cb092d8 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.297314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 442/878] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:43.303231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9cd88-ee1f-4f5e-8054-22f40b9e8e24 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.303584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9cd88-ee1f-4f5e-8054-22f40b9e8e24 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.303879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9cd88-ee1f-4f5e-8054-22f40b9e8e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.322089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-28b9cd88-ee1f-4f5e-8054-22f40b9e8e24 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:43.323310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-28b9cd88-ee1f-4f5e-8054-22f40b9e8e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:43.388693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa64ea6-3387-40c4-83f6-6a4b488ae06d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:43.408506 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 443/879] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:54:43.426965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabaf085-8f2b-4d13-979a-dbb328bf12ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:43.428133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabaf085-8f2b-4d13-979a-dbb328bf12ec None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.428635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabaf085-8f2b-4d13-979a-dbb328bf12ec None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.429213 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabaf085-8f2b-4d13-979a-dbb328bf12ec None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.430182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabaf085-8f2b-4d13-979a-dbb328bf12ec None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:43.430706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabaf085-8f2b-4d13-979a-dbb328bf12ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:43.432369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9cd88-ee1f-4f5e-8054-22f40b9e8e24 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:43.432703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabaf085-8f2b-4d13-979a-dbb328bf12ec None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:43.444388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 437/880] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:54:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:43.445830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabaf085-8f2b-4d13-979a-dbb328bf12ec None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:43.456444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:43.476200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:43.547784 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9cd88-ee1f-4f5e-8054-22f40b9e8e24 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:43.574226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabaf085-8f2b-4d13-979a-dbb328bf12ec None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:43.588472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e61ae-b6b0-445c-ac73-f058fe66de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:43.589393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e61ae-b6b0-445c-ac73-f058fe66de4c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.589614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e61ae-b6b0-445c-ac73-f058fe66de4c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.589894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e61ae-b6b0-445c-ac73-f058fe66de4c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.602601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e61ae-b6b0-445c-ac73-f058fe66de4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c59e61ae-b6b0-445c-ac73-f058fe66de4c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:43.611205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:43.612321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:43.613160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 444/881] 10.222.0.75 () {56 vars in 1120 bytes} [Thu Jul 14 10:54:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:43.618161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e61ae-b6b0-445c-ac73-f058fe66de4c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:43.621023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21597809-1f6f-427e-b629-d5596068fd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:43.621699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21597809-1f6f-427e-b629-d5596068fd37 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.621982 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21597809-1f6f-427e-b629-d5596068fd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.622258 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21597809-1f6f-427e-b629-d5596068fd37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.623046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21597809-1f6f-427e-b629-d5596068fd37 None admin] RBAC: Adding query filter params (interface=public, service_id=e369ce8d2cef4a39aa993038d659482e, region_id=RegionOne) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:43.623360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21597809-1f6f-427e-b629-d5596068fd37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:43.624764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21597809-1f6f-427e-b629-d5596068fd37 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:43.635053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 438/882] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Jul 14 10:54:43 2022] GET /identity/v3/endpoints?service_id=e369ce8d2cef4a39aa993038d659482e&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:43.637180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21597809-1f6f-427e-b629-d5596068fd37 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:43.732614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:43.749186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:43.754037 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e61ae-b6b0-445c-ac73-f058fe66de4c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:43.772760 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21597809-1f6f-427e-b629-d5596068fd37 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:43.793650 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9683150-d02d-46f5-8d0c-d502d3b129cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:43.794486 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9683150-d02d-46f5-8d0c-d502d3b129cb None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.794869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9683150-d02d-46f5-8d0c-d502d3b129cb None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.795561 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9683150-d02d-46f5-8d0c-d502d3b129cb None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.796229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9683150-d02d-46f5-8d0c-d502d3b129cb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:43.796653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9683150-d02d-46f5-8d0c-d502d3b129cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:43.798410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9683150-d02d-46f5-8d0c-d502d3b129cb None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:43.807015 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 445/883] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Jul 14 10:54:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:43.808611 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312e0e-5aed-4ea7-a7e3-36d0a283d736 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.808903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312e0e-5aed-4ea7-a7e3-36d0a283d736 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.809184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312e0e-5aed-4ea7-a7e3-36d0a283d736 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.810536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 446/884] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:43.814075 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64312e0e-5aed-4ea7-a7e3-36d0a283d736 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:43.818498 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d878b-5964-41ae-bdfa-fa018ede0cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:43.820056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d878b-5964-41ae-bdfa-fa018ede0cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.820352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d878b-5964-41ae-bdfa-fa018ede0cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.820636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d878b-5964-41ae-bdfa-fa018ede0cf6 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.821338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d878b-5964-41ae-bdfa-fa018ede0cf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:43.822729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d878b-5964-41ae-bdfa-fa018ede0cf6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:43.829068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 439/885] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Jul 14 10:54:43 2022] GET /identity/v3/services => generated 1011 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:54:43.830380 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb9d68-8209-4147-8aa5-7fb459f39ac9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.830681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb9d68-8209-4147-8aa5-7fb459f39ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.830967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb9d68-8209-4147-8aa5-7fb459f39ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.850273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-54bb9d68-8209-4147-8aa5-7fb459f39ac9 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:43.851694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-54bb9d68-8209-4147-8aa5-7fb459f39ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:43.931361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 440/886] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:43.933309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254832a8-9c18-4491-bb5a-3a2980ede8ca None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.933783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254832a8-9c18-4491-bb5a-3a2980ede8ca None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.934237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254832a8-9c18-4491-bb5a-3a2980ede8ca None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.936011 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 441/887] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:43.942214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79809d7e-c38f-40f9-ae78-6464be67c8df None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.942528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79809d7e-c38f-40f9-ae78-6464be67c8df None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.942816 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79809d7e-c38f-40f9-ae78-6464be67c8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:43.955708 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64312e0e-5aed-4ea7-a7e3-36d0a283d736 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:43.966248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-79809d7e-c38f-40f9-ae78-6464be67c8df None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:43.967570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-79809d7e-c38f-40f9-ae78-6464be67c8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:43.995127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e60ae-5f81-4db6-92b8-dc32b5575592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:43.995967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e60ae-5f81-4db6-92b8-dc32b5575592 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:43.996244 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e60ae-5f81-4db6-92b8-dc32b5575592 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:43.996519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e60ae-5f81-4db6-92b8-dc32b5575592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:44.009455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e60ae-5f81-4db6-92b8-dc32b5575592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:44.010944 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e60ae-5f81-4db6-92b8-dc32b5575592 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:44.014504 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-ba9e60ae-5f81-4db6-92b8-dc32b5575592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:44.035207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:44.036296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:44.037379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:44.037379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:44.037379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:44.037379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:44.037379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 447/888] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:44.037721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e60ae-5f81-4db6-92b8-dc32b5575592 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:44.052472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 442/889] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:54:44.053448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79809d7e-c38f-40f9-ae78-6464be67c8df None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:44.077372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:44.178605 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e60ae-5f81-4db6-92b8-dc32b5575592 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:44.215929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79809d7e-c38f-40f9-ae78-6464be67c8df None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:44.230572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc3811-6c47-4879-95db-998397e66c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:44.231251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc3811-6c47-4879-95db-998397e66c5f None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:44.231583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc3811-6c47-4879-95db-998397e66c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:44.231866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc3811-6c47-4879-95db-998397e66c5f None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:44.232590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc3811-6c47-4879-95db-998397e66c5f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:44.233964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc3811-6c47-4879-95db-998397e66c5f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:44.249481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 448/890] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:54:44 2022] POST /identity/v3/services => generated 269 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:54:44.250161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc3811-6c47-4879-95db-998397e66c5f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:44.294613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc4d5e-eebc-4be8-babc-4e6235dad2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:44.297098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc4d5e-eebc-4be8-babc-4e6235dad2bd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:44.297759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc4d5e-eebc-4be8-babc-4e6235dad2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:44.298262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc4d5e-eebc-4be8-babc-4e6235dad2bd None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:44.299303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc4d5e-eebc-4be8-babc-4e6235dad2bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:44.299867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc4d5e-eebc-4be8-babc-4e6235dad2bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:44.301679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc4d5e-eebc-4be8-babc-4e6235dad2bd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:44.322292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 443/891] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:44.326129 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:44.328779 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc4d5e-eebc-4be8-babc-4e6235dad2bd None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:44.386002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc3811-6c47-4879-95db-998397e66c5f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:44.405862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:44.429317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0f964-edf1-4d5e-820c-ef45eca24df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:44.430146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f964-edf1-4d5e-820c-ef45eca24df0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:44.430429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f964-edf1-4d5e-820c-ef45eca24df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:44.430705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f964-edf1-4d5e-820c-ef45eca24df0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:44.431368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0f964-edf1-4d5e-820c-ef45eca24df0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:44.432821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0f964-edf1-4d5e-820c-ef45eca24df0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-edd0f964-edf1-4d5e-820c-ef45eca24df0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:44.446575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:44.447741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:44.449191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 449/892] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Jul 14 10:54:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:44.456781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0f964-edf1-4d5e-820c-ef45eca24df0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:44.457750 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc4d5e-eebc-4be8-babc-4e6235dad2bd None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:44.503351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04c483-4730-4059-8147-98a247a0ee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:44.504862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04c483-4730-4059-8147-98a247a0ee6b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:44.505161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04c483-4730-4059-8147-98a247a0ee6b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:44.505447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04c483-4730-4059-8147-98a247a0ee6b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:44.514095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee04c483-4730-4059-8147-98a247a0ee6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:44.515604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee04c483-4730-4059-8147-98a247a0ee6b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-ee04c483-4730-4059-8147-98a247a0ee6b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:44.532798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:44.533811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:44.534747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:44.534747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:44.534747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:44.534747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:44.534747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:44.534747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:44.534747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:44.534747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:44.534747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:54:44.534747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:44.542274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 444/893] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Jul 14 10:54:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:44.544301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04c483-4730-4059-8147-98a247a0ee6b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:44.586759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0f964-edf1-4d5e-820c-ef45eca24df0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:44.606427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:44.626529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10ecec-0cb5-4394-9b00-f4230496b723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:44.627321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10ecec-0cb5-4394-9b00-f4230496b723 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:44.627714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10ecec-0cb5-4394-9b00-f4230496b723 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:44.628070 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10ecec-0cb5-4394-9b00-f4230496b723 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:44.628856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10ecec-0cb5-4394-9b00-f4230496b723 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:44.629813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10ecec-0cb5-4394-9b00-f4230496b723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:44.631269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10ecec-0cb5-4394-9b00-f4230496b723 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:44.640997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 450/894] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Jul 14 10:54:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:44.642316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10ecec-0cb5-4394-9b00-f4230496b723 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:44.687663 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04c483-4730-4059-8147-98a247a0ee6b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:44.716620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:44.730526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f00bb0-1ef5-46dd-bed4-fd185b5338be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:44.731309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f00bb0-1ef5-46dd-bed4-fd185b5338be None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:44.731695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f00bb0-1ef5-46dd-bed4-fd185b5338be None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:44.732061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f00bb0-1ef5-46dd-bed4-fd185b5338be None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:44.732994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f00bb0-1ef5-46dd-bed4-fd185b5338be None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:44.733373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f00bb0-1ef5-46dd-bed4-fd185b5338be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:44.735084 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f00bb0-1ef5-46dd-bed4-fd185b5338be None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:44.761575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 445/895] 10.222.0.75 () {56 vars in 1149 bytes} [Thu Jul 14 10:54:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:44.762960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f00bb0-1ef5-46dd-bed4-fd185b5338be None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:44.783370 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10ecec-0cb5-4394-9b00-f4230496b723 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:44.833337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c2825-29ff-45ff-85e0-7fe2df19922e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:44.834992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c2825-29ff-45ff-85e0-7fe2df19922e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:44.836113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c2825-29ff-45ff-85e0-7fe2df19922e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:44.836582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c2825-29ff-45ff-85e0-7fe2df19922e None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:44.837826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c2825-29ff-45ff-85e0-7fe2df19922e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:44.838306 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c2825-29ff-45ff-85e0-7fe2df19922e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:44.841147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c2825-29ff-45ff-85e0-7fe2df19922e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:44.848338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 451/896] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Jul 14 10:54:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:44.885375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f00bb0-1ef5-46dd-bed4-fd185b5338be None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:44.926944 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4669bf-5303-4271-9e88-e8606e9af17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:44.927644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4669bf-5303-4271-9e88-e8606e9af17d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:44.927930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4669bf-5303-4271-9e88-e8606e9af17d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:44.928211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4669bf-5303-4271-9e88-e8606e9af17d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:44.989943 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:45.014465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4669bf-5303-4271-9e88-e8606e9af17d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:45.016421 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4669bf-5303-4271-9e88-e8606e9af17d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:45.130866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 446/897] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:44 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:54:45.132269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c2825-29ff-45ff-85e0-7fe2df19922e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:45.183673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:45.242256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c2825-29ff-45ff-85e0-7fe2df19922e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:45.277721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d509a-0ab3-4bae-9023-9ae455b64581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:45.278534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d509a-0ab3-4bae-9023-9ae455b64581 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:45.278825 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d509a-0ab3-4bae-9023-9ae455b64581 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:45.279110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d509a-0ab3-4bae-9023-9ae455b64581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:45.291169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d509a-0ab3-4bae-9023-9ae455b64581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:45.292678 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d509a-0ab3-4bae-9023-9ae455b64581 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-011d509a-0ab3-4bae-9023-9ae455b64581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:45.310791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:45.311886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:45.312763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:45.312763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:45.312763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:45.312763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:45.312763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 452/898] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:45.317747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4669bf-5303-4271-9e88-e8606e9af17d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:45.406874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:45.430826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4669bf-5303-4271-9e88-e8606e9af17d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:45.470665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b0579-4326-4cf2-aa0a-67f04f7f8a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:45.471354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b0579-4326-4cf2-aa0a-67f04f7f8a44 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:45.471670 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b0579-4326-4cf2-aa0a-67f04f7f8a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:45.471955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b0579-4326-4cf2-aa0a-67f04f7f8a44 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:45.472734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b0579-4326-4cf2-aa0a-67f04f7f8a44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:45.473049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b0579-4326-4cf2-aa0a-67f04f7f8a44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:45.474454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b0579-4326-4cf2-aa0a-67f04f7f8a44 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:45.487305 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 447/899] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:45.495278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d509a-0ab3-4bae-9023-9ae455b64581 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:45.601369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:45.606902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d509a-0ab3-4bae-9023-9ae455b64581 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:45.643154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a6ed6-b4af-4792-a618-b1312947876e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:45.644077 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a6ed6-b4af-4792-a618-b1312947876e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:45.644364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a6ed6-b4af-4792-a618-b1312947876e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:45.645224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a6ed6-b4af-4792-a618-b1312947876e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:45.652629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a6ed6-b4af-4792-a618-b1312947876e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:45.654088 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a6ed6-b4af-4792-a618-b1312947876e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a51a6ed6-b4af-4792-a618-b1312947876e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:45.668877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:45.670925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:45.673116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:45.673116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:45.673116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:45.673116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:45.673116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:45.673116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:45.673116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:45.673116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:45.679418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 453/900] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:45.679690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b0579-4326-4cf2-aa0a-67f04f7f8a44 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:45.783716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b0579-4326-4cf2-aa0a-67f04f7f8a44 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:45.819693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b969f38-6e5d-4f3d-a897-876655dba17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:45.820373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b969f38-6e5d-4f3d-a897-876655dba17a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:45.820657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b969f38-6e5d-4f3d-a897-876655dba17a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:45.820939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b969f38-6e5d-4f3d-a897-876655dba17a None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:45.821736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b969f38-6e5d-4f3d-a897-876655dba17a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:45.822043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b969f38-6e5d-4f3d-a897-876655dba17a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:45.823525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b969f38-6e5d-4f3d-a897-876655dba17a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:45.837597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 448/901] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:45.843352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a6ed6-b4af-4792-a618-b1312947876e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:45.957127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a6ed6-b4af-4792-a618-b1312947876e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:45.972715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086b08d-c859-4336-a986-6eda165974b1 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:45.973290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086b08d-c859-4336-a986-6eda165974b1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:45.973505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086b08d-c859-4336-a986-6eda165974b1 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:45.975252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 449/902] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:45.980679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb2d7d-560c-41a2-91b0-b17e499f73b5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:45.981108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb2d7d-560c-41a2-91b0-b17e499f73b5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:45.981512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb2d7d-560c-41a2-91b0-b17e499f73b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:45.986707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:45.999438 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a046e8-2626-447e-99ae-9bf8d28d2c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:46.000386 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a046e8-2626-447e-99ae-9bf8d28d2c55 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:46.000826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a046e8-2626-447e-99ae-9bf8d28d2c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:46.001250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a046e8-2626-447e-99ae-9bf8d28d2c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:46.002253 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a046e8-2626-447e-99ae-9bf8d28d2c55 None admin] RBAC: Adding query filter params (role.id=4d8a62dacbc5492b9fa4b11c57673134, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=f40c0fcc93cf4bbf8d35c1de70d087ba) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:46.002826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a046e8-2626-447e-99ae-9bf8d28d2c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:46.003644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bfeb2d7d-560c-41a2-91b0-b17e499f73b5 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:46.005231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a046e8-2626-447e-99ae-9bf8d28d2c55 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:46.005632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bfeb2d7d-560c-41a2-91b0-b17e499f73b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:46.019529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 454/903] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:45 2022] GET /identity/v3/role_assignments?user.id=f40c0fcc93cf4bbf8d35c1de70d087ba&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=4d8a62dacbc5492b9fa4b11c57673134 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:46.091566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 450/904] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:54:46.115240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a046e8-2626-447e-99ae-9bf8d28d2c55 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:46.151189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:46.247781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a046e8-2626-447e-99ae-9bf8d28d2c55 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:46.283403 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3454ca-7c8b-4cea-9700-353efc3f40a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:46.284545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3454ca-7c8b-4cea-9700-353efc3f40a7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:46.285022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3454ca-7c8b-4cea-9700-353efc3f40a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:46.286012 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3454ca-7c8b-4cea-9700-353efc3f40a7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:46.286861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3454ca-7c8b-4cea-9700-353efc3f40a7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:46.288447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3454ca-7c8b-4cea-9700-353efc3f40a7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-0a3454ca-7c8b-4cea-9700-353efc3f40a7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:46.301035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:46.302562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:54:46.303373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:46.305954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 455/905] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:54:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:46.312766 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb2d7d-560c-41a2-91b0-b17e499f73b5 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:46.392323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc98640-cb2d-4a2e-b0b8-c289661c56b8 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:46.392831 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc98640-cb2d-4a2e-b0b8-c289661c56b8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:46.393399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc98640-cb2d-4a2e-b0b8-c289661c56b8 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:46.395027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 456/906] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:54:46.400509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f6ba9-0c06-44c3-8c02-33a245915cd0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:46.401054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f6ba9-0c06-44c3-8c02-33a245915cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:46.401331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f6ba9-0c06-44c3-8c02-33a245915cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:46.422602 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb2d7d-560c-41a2-91b0-b17e499f73b5 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:46.424024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5a8f6ba9-0c06-44c3-8c02-33a245915cd0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:46.426898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5a8f6ba9-0c06-44c3-8c02-33a245915cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:46.464875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:46.471069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1b0c5-9494-469b-af16-f7f41f638754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:46.471867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1b0c5-9494-469b-af16-f7f41f638754 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:46.472217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1b0c5-9494-469b-af16-f7f41f638754 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:46.473684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1b0c5-9494-469b-af16-f7f41f638754 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:46.474476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1b0c5-9494-469b-af16-f7f41f638754 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:46.474853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1b0c5-9494-469b-af16-f7f41f638754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:46.477284 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1b0c5-9494-469b-af16-f7f41f638754 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:46.484869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 451/907] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:54:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:46.492941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1b0c5-9494-469b-af16-f7f41f638754 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:46.511685 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 457/908] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:46.530459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f6ba9-0c06-44c3-8c02-33a245915cd0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:46.549579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:46.618678 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1b0c5-9494-469b-af16-f7f41f638754 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:46.654494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e3b90-5fb9-4964-9aab-b2136dd1f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:46.655158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e3b90-5fb9-4964-9aab-b2136dd1f90c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:46.655465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e3b90-5fb9-4964-9aab-b2136dd1f90c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:46.655745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e3b90-5fb9-4964-9aab-b2136dd1f90c None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:46.656515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e3b90-5fb9-4964-9aab-b2136dd1f90c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:46.657114 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e3b90-5fb9-4964-9aab-b2136dd1f90c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:46.658186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e3b90-5fb9-4964-9aab-b2136dd1f90c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:46.668772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 452/909] 10.222.0.75 () {56 vars in 1114 bytes} [Thu Jul 14 10:54:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:46.669864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e3b90-5fb9-4964-9aab-b2136dd1f90c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:46.676504 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f6ba9-0c06-44c3-8c02-33a245915cd0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:46.715110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75e101-9c4a-4910-aedf-407a9fb09063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:46.716086 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75e101-9c4a-4910-aedf-407a9fb09063 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:46.716445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75e101-9c4a-4910-aedf-407a9fb09063 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:46.716807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75e101-9c4a-4910-aedf-407a9fb09063 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:46.717580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75e101-9c4a-4910-aedf-407a9fb09063 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:46.719082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75e101-9c4a-4910-aedf-407a9fb09063 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-3a75e101-9c4a-4910-aedf-407a9fb09063 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:46.732659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:46.733666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:54:46.734499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:46.739026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 458/910] 10.222.0.75 () {56 vars in 1132 bytes} [Thu Jul 14 10:54:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:46.740247 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75e101-9c4a-4910-aedf-407a9fb09063 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:46.758681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:46.806836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e3b90-5fb9-4964-9aab-b2136dd1f90c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:46.846251 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af053562-76f0-4261-af8a-ae2270afcfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:46.846912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af053562-76f0-4261-af8a-ae2270afcfe2 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:46.847414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af053562-76f0-4261-af8a-ae2270afcfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:46.847768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:46.848676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af053562-76f0-4261-af8a-ae2270afcfe2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:46.850120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af053562-76f0-4261-af8a-ae2270afcfe2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:46.852175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af053562-76f0-4261-af8a-ae2270afcfe2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:46.874020 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75e101-9c4a-4910-aedf-407a9fb09063 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:46.883425 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 453/911] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Jul 14 10:54:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:54:46.933787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32bdd4-eab7-4a73-9015-61d0a8ee5616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:46.934475 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32bdd4-eab7-4a73-9015-61d0a8ee5616 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:46.934791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32bdd4-eab7-4a73-9015-61d0a8ee5616 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:46.935027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32bdd4-eab7-4a73-9015-61d0a8ee5616 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:46.935901 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32bdd4-eab7-4a73-9015-61d0a8ee5616 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:46.937422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32bdd4-eab7-4a73-9015-61d0a8ee5616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:46.937700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32bdd4-eab7-4a73-9015-61d0a8ee5616 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:46.944734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 459/912] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Jul 14 10:54:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:46.950522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af053562-76f0-4261-af8a-ae2270afcfe2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:47.068961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af053562-76f0-4261-af8a-ae2270afcfe2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:47.087186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:47.109510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883b722-96e9-48b8-8eb3-01735beee20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:47.110549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883b722-96e9-48b8-8eb3-01735beee20d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:47.111091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883b722-96e9-48b8-8eb3-01735beee20d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:47.111606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883b722-96e9-48b8-8eb3-01735beee20d None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:47.112540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883b722-96e9-48b8-8eb3-01735beee20d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:47.113056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883b722-96e9-48b8-8eb3-01735beee20d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:47.115214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883b722-96e9-48b8-8eb3-01735beee20d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:47.121314 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 454/913] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Jul 14 10:54:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:47.127322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32bdd4-eab7-4a73-9015-61d0a8ee5616 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:47.135012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186990b-59c2-4aed-88c2-39b327fb0ad8 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:47.135308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186990b-59c2-4aed-88c2-39b327fb0ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:47.135615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186990b-59c2-4aed-88c2-39b327fb0ad8 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:47.137261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 455/914] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:47.142521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fd7a3-7bcd-4479-bb7c-4c63a39f0dd0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:47.142828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fd7a3-7bcd-4479-bb7c-4c63a39f0dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:47.143105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fd7a3-7bcd-4479-bb7c-4c63a39f0dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:47.162610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-3f4fd7a3-7bcd-4479-bb7c-4c63a39f0dd0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:47.164794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3f4fd7a3-7bcd-4479-bb7c-4c63a39f0dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:47.249232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32bdd4-eab7-4a73-9015-61d0a8ee5616 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:47.249561 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 456/915] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:54:47.275370 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fd7a3-7bcd-4479-bb7c-4c63a39f0dd0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:47.286986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a63fac-4839-49e7-b10d-3f3bb9caf4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:47.288031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a63fac-4839-49e7-b10d-3f3bb9caf4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:47.288567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a63fac-4839-49e7-b10d-3f3bb9caf4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:47.289036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a63fac-4839-49e7-b10d-3f3bb9caf4d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:47.290017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a63fac-4839-49e7-b10d-3f3bb9caf4d4 None admin] RBAC: Adding query filter params (interface=public, service_id=2310928f28d6493d85ddc158260edaf1, region_id=RegionOne) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:47.290525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a63fac-4839-49e7-b10d-3f3bb9caf4d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:47.292350 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a63fac-4839-49e7-b10d-3f3bb9caf4d4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:47.301868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 460/916] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Jul 14 10:54:47 2022] GET /identity/v3/endpoints?service_id=2310928f28d6493d85ddc158260edaf1&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:47.307414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a63fac-4839-49e7-b10d-3f3bb9caf4d4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:47.311168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:47.388050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fd7a3-7bcd-4479-bb7c-4c63a39f0dd0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:47.401326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:47.434090 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fd16c-3566-44e8-9df0-94f7a9168787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:47.434768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fd16c-3566-44e8-9df0-94f7a9168787 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:47.435055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fd16c-3566-44e8-9df0-94f7a9168787 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:47.435336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fd16c-3566-44e8-9df0-94f7a9168787 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:47.436099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fd16c-3566-44e8-9df0-94f7a9168787 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:47.437482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fd16c-3566-44e8-9df0-94f7a9168787 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:47.459886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 457/917] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:54:47 2022] POST /identity/v3/services => generated 241 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:54:47.460896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a63fac-4839-49e7-b10d-3f3bb9caf4d4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:47.508587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32c994-86f5-4f09-a2f1-07407baf24a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:47.509714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32c994-86f5-4f09-a2f1-07407baf24a2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:47.510290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32c994-86f5-4f09-a2f1-07407baf24a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:47.510788 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32c994-86f5-4f09-a2f1-07407baf24a2 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:47.511796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32c994-86f5-4f09-a2f1-07407baf24a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:47.513556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32c994-86f5-4f09-a2f1-07407baf24a2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:47.521813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 461/918] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Jul 14 10:54:47 2022] GET /identity/v3/services => generated 1497 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:54:48.141855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869f13e-a3cc-4817-958f-81042cf321dc None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:48.142403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869f13e-a3cc-4817-958f-81042cf321dc None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:48.142403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869f13e-a3cc-4817-958f-81042cf321dc None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:48.144219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 458/919] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:48.149859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db2699-1f98-4786-8c12-13023bc51b21 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:48.150377 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db2699-1f98-4786-8c12-13023bc51b21 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:48.150464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db2699-1f98-4786-8c12-13023bc51b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:48.167534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d1db2699-1f98-4786-8c12-13023bc51b21 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:48.168841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d1db2699-1f98-4786-8c12-13023bc51b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:48.239722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:48.249063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 462/920] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:48.272924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9869f13e-a3cc-4817-958f-81042cf321dc None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:48.389581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9869f13e-a3cc-4817-958f-81042cf321dc None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:48.425540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497537c6-5807-449f-b460-06f1f1af13f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:48.428569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497537c6-5807-449f-b460-06f1f1af13f5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:48.428861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497537c6-5807-449f-b460-06f1f1af13f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:48.429143 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497537c6-5807-449f-b460-06f1f1af13f5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:48.441076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497537c6-5807-449f-b460-06f1f1af13f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-497537c6-5807-449f-b460-06f1f1af13f5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:48.455581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:48.457686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:48.458819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 459/921] 10.222.0.75 () {56 vars in 1120 bytes} [Thu Jul 14 10:54:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:48.466731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db2699-1f98-4786-8c12-13023bc51b21 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:48.588090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db2699-1f98-4786-8c12-13023bc51b21 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:48.629499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735363e-81a7-4186-b69a-5b3c0876e4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:48.630249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735363e-81a7-4186-b69a-5b3c0876e4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:48.630755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735363e-81a7-4186-b69a-5b3c0876e4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:48.631253 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735363e-81a7-4186-b69a-5b3c0876e4f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:48.632660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735363e-81a7-4186-b69a-5b3c0876e4f4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:48.633211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735363e-81a7-4186-b69a-5b3c0876e4f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:48.634807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735363e-81a7-4186-b69a-5b3c0876e4f4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:48.645907 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 463/922] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Jul 14 10:54:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:48.652847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497537c6-5807-449f-b460-06f1f1af13f5 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:48.703946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:48.784993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497537c6-5807-449f-b460-06f1f1af13f5 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:48.823026 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65e917-0b1c-42ca-b5b6-8ea7be079af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:48.824285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65e917-0b1c-42ca-b5b6-8ea7be079af4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:48.824776 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65e917-0b1c-42ca-b5b6-8ea7be079af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:48.825268 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65e917-0b1c-42ca-b5b6-8ea7be079af4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:48.838396 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65e917-0b1c-42ca-b5b6-8ea7be079af4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:48.840229 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65e917-0b1c-42ca-b5b6-8ea7be079af4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-cc65e917-0b1c-42ca-b5b6-8ea7be079af4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:48.860064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:48.861096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:48.861973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:48.861973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:48.861973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:48.861973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:48.869384 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 460/923] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:48.870963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735363e-81a7-4186-b69a-5b3c0876e4f4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:48.946053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:48.998711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735363e-81a7-4186-b69a-5b3c0876e4f4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:49.037402 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10073ba5-57a7-4d29-8470-a9def252c64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:49.038084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10073ba5-57a7-4d29-8470-a9def252c64f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.038454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10073ba5-57a7-4d29-8470-a9def252c64f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.038954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10073ba5-57a7-4d29-8470-a9def252c64f None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.040384 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10073ba5-57a7-4d29-8470-a9def252c64f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:49.040874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10073ba5-57a7-4d29-8470-a9def252c64f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:49.042755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10073ba5-57a7-4d29-8470-a9def252c64f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:49.057185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 464/924] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:49.063488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65e917-0b1c-42ca-b5b6-8ea7be079af4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:49.171066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:49.199372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65e917-0b1c-42ca-b5b6-8ea7be079af4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:49.205407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc7cca-4696-4636-87a6-a8881b2b775f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.206811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc7cca-4696-4636-87a6-a8881b2b775f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.207927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc7cca-4696-4636-87a6-a8881b2b775f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.210785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 465/925] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:49.215989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77733b62-a546-4e67-8841-6a52de4539dc None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.216287 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77733b62-a546-4e67-8841-6a52de4539dc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.216577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77733b62-a546-4e67-8841-6a52de4539dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.234465 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-77733b62-a546-4e67-8841-6a52de4539dc None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:49.235694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-77733b62-a546-4e67-8841-6a52de4539dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:49.244461 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde23f9-b0bc-4f96-83e4-b8404ba069aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:49.245757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde23f9-b0bc-4f96-83e4-b8404ba069aa None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.246264 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde23f9-b0bc-4f96-83e4-b8404ba069aa None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.246793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde23f9-b0bc-4f96-83e4-b8404ba069aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.257051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde23f9-b0bc-4f96-83e4-b8404ba069aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:49.259587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde23f9-b0bc-4f96-83e4-b8404ba069aa None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-9bde23f9-b0bc-4f96-83e4-b8404ba069aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:49.279867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:49.281004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:49.282551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:49.282551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:49.282551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:49.282551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:49.282551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:49.282551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:49.282551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:49.282551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:49.282551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:54:49.282551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:49.291219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 461/926] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Jul 14 10:54:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:49.292856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde23f9-b0bc-4f96-83e4-b8404ba069aa None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:49.323738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 466/927] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:54:49.342137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77733b62-a546-4e67-8841-6a52de4539dc None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:49.403660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde23f9-b0bc-4f96-83e4-b8404ba069aa None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:49.427780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:49.440995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a450091-4198-4c99-b3a9-56242117d23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:49.442583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a450091-4198-4c99-b3a9-56242117d23c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.443079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a450091-4198-4c99-b3a9-56242117d23c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.443606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a450091-4198-4c99-b3a9-56242117d23c None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.444733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a450091-4198-4c99-b3a9-56242117d23c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:49.445315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a450091-4198-4c99-b3a9-56242117d23c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:49.447069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a450091-4198-4c99-b3a9-56242117d23c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:49.454039 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77733b62-a546-4e67-8841-6a52de4539dc None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:49.464192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:49.473634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 462/928] 10.222.0.75 () {56 vars in 1149 bytes} [Thu Jul 14 10:54:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:49.474983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a450091-4198-4c99-b3a9-56242117d23c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:49.498404 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c1368-5634-4428-b08c-04baa4ea66c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:49.499493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c1368-5634-4428-b08c-04baa4ea66c3 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.500002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c1368-5634-4428-b08c-04baa4ea66c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.500482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c1368-5634-4428-b08c-04baa4ea66c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.501549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c1368-5634-4428-b08c-04baa4ea66c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:49.503185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c1368-5634-4428-b08c-04baa4ea66c3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:49.505709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: WARNING keystone.server.flask.common [None req-960c1368-5634-4428-b08c-04baa4ea66c3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 10:54:49.525583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.sql.core [None req-960c1368-5634-4428-b08c-04baa4ea66c3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 10:54:49.525583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 10:54:49.525583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [parameters: {'id': 'f9469fa1e60e4505a71db0103adb71d2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 10:54:49.525583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80183) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-960c1368-5634-4428-b08c-04baa4ea66c3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f9469fa1e60e4505a71db0103adb71d2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f9469fa1e60e4505a71db0103adb71d2. Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:54:49.534803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result.read() Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 14 10:54:49.535768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application yield resource Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise self.value Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application session.commit() Jul 14 10:54:49.536727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.session.flush() Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application compat.raise_( Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 10:54:49.537612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 10:54:49.538509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result.read() Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:54:49.539295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [parameters: {'id': 'f9469fa1e60e4505a71db0103adb71d2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:49.540181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:49.540963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:49.541962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:49.541962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 14 10:54:49.541962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 14 10:54:49.541962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:49.541962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:49.541962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 14 10:54:49.541962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 14 10:54:49.541962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f9469fa1e60e4505a71db0103adb71d2. Jul 14 10:54:49.541962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:49.550616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 467/929] 10.222.0.75 () {60 vars in 1140 bytes} [Thu Jul 14 10:54:49 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:49.552958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c1368-5634-4428-b08c-04baa4ea66c3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:49.585651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a450091-4198-4c99-b3a9-56242117d23c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:49.622193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6ad76-04c3-4eee-ba15-2d2bf7201cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:49.623016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6ad76-04c3-4eee-ba15-2d2bf7201cef None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.623298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6ad76-04c3-4eee-ba15-2d2bf7201cef None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.623606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6ad76-04c3-4eee-ba15-2d2bf7201cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.635190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6ad76-04c3-4eee-ba15-2d2bf7201cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:49.636790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6ad76-04c3-4eee-ba15-2d2bf7201cef None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-57c6ad76-04c3-4eee-ba15-2d2bf7201cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:49.655899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:49.656854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:49.657746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:49.657746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:49.657746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:49.657746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:49.662545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 463/930] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:49.664212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6ad76-04c3-4eee-ba15-2d2bf7201cef None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:49.674917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c1368-5634-4428-b08c-04baa4ea66c3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:49.682665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:49.715397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb61124-9448-4fc5-b6c7-5f2bcff6813b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:49.716590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb61124-9448-4fc5-b6c7-5f2bcff6813b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.716975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb61124-9448-4fc5-b6c7-5f2bcff6813b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.717150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb61124-9448-4fc5-b6c7-5f2bcff6813b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.723682 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:49.725466 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb61124-9448-4fc5-b6c7-5f2bcff6813b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:49.727140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb61124-9448-4fc5-b6c7-5f2bcff6813b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-dcb61124-9448-4fc5-b6c7-5f2bcff6813b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:49.741440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:49.742594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:49.743763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:49.743763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:49.743763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:49.743763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:49.743763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:49.743763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:49.743763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:49.743763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:49.744501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 468/931] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:49.745828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533ec46-3820-44f0-9599-7cb6e0e43bea None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.746115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533ec46-3820-44f0-9599-7cb6e0e43bea None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.746401 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533ec46-3820-44f0-9599-7cb6e0e43bea None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.754647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 469/932] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:49 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:49.755513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533ec46-3820-44f0-9599-7cb6e0e43bea None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:49.790270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6ad76-04c3-4eee-ba15-2d2bf7201cef None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:49.823898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1bf7d-0a99-48fe-a87a-c7f2bdc5297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:49.824568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1bf7d-0a99-48fe-a87a-c7f2bdc5297d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.824856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1bf7d-0a99-48fe-a87a-c7f2bdc5297d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.825138 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1bf7d-0a99-48fe-a87a-c7f2bdc5297d None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.825863 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1bf7d-0a99-48fe-a87a-c7f2bdc5297d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:49.826175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1bf7d-0a99-48fe-a87a-c7f2bdc5297d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:49.827587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1bf7d-0a99-48fe-a87a-c7f2bdc5297d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:49.839752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 464/933] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:49.841285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be88c2-5789-42a7-a995-22a6a1fc2a9c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.841579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be88c2-5789-42a7-a995-22a6a1fc2a9c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.841857 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be88c2-5789-42a7-a995-22a6a1fc2a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.858962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b3be88c2-5789-42a7-a995-22a6a1fc2a9c None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:49.860192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b3be88c2-5789-42a7-a995-22a6a1fc2a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:49.865205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533ec46-3820-44f0-9599-7cb6e0e43bea None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:49.899277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83388635-dadf-49a9-b590-9f7739db0f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:49.899974 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83388635-dadf-49a9-b590-9f7739db0f2c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.900263 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83388635-dadf-49a9-b590-9f7739db0f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.900537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83388635-dadf-49a9-b590-9f7739db0f2c None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.901298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83388635-dadf-49a9-b590-9f7739db0f2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:49.901605 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83388635-dadf-49a9-b590-9f7739db0f2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:49.903035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83388635-dadf-49a9-b590-9f7739db0f2c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:49.921344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 470/934] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:54:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:49.922762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec68a5-02f8-4dd3-a6fa-ccbe8f3d2163 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.923059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec68a5-02f8-4dd3-a6fa-ccbe8f3d2163 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.923341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec68a5-02f8-4dd3-a6fa-ccbe8f3d2163 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.924628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 471/935] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:49.925351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec68a5-02f8-4dd3-a6fa-ccbe8f3d2163 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:49.937941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 465/936] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:49.939785 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead2833-a529-4ff8-bb28-7603adafeae7 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:49.940332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead2833-a529-4ff8-bb28-7603adafeae7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:49.940903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead2833-a529-4ff8-bb28-7603adafeae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:49.963509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:49.969474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7ead2833-a529-4ff8-bb28-7603adafeae7 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:49.970825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7ead2833-a529-4ff8-bb28-7603adafeae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:50.012240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:50.048871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 466/937] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:50.049690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead2833-a529-4ff8-bb28-7603adafeae7 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:50.065905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec68a5-02f8-4dd3-a6fa-ccbe8f3d2163 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:50.109001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557a145-fb1c-4322-8a26-8f390139761e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:50.109969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557a145-fb1c-4322-8a26-8f390139761e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.110362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557a145-fb1c-4322-8a26-8f390139761e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.110654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557a145-fb1c-4322-8a26-8f390139761e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.118493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557a145-fb1c-4322-8a26-8f390139761e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:50.120395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557a145-fb1c-4322-8a26-8f390139761e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-4557a145-fb1c-4322-8a26-8f390139761e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:50.136364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:50.137483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:50.138390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:50.138390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:50.138390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:50.138390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:50.138390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:50.138390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:50.138390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:50.138390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:50.138390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 472/938] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:50.138390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557a145-fb1c-4322-8a26-8f390139761e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:50.175314 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead2833-a529-4ff8-bb28-7603adafeae7 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:50.218736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08231ee-d47a-4bad-8d9c-1e7ebaaf331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:50.219693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08231ee-d47a-4bad-8d9c-1e7ebaaf331c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.219977 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08231ee-d47a-4bad-8d9c-1e7ebaaf331c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.220255 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08231ee-d47a-4bad-8d9c-1e7ebaaf331c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.220909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08231ee-d47a-4bad-8d9c-1e7ebaaf331c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:50.222285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08231ee-d47a-4bad-8d9c-1e7ebaaf331c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-b08231ee-d47a-4bad-8d9c-1e7ebaaf331c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:50.235134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:50.236202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:50.237722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:50.238729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:50.239143 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 467/939] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Jul 14 10:54:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:50.239143 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08231ee-d47a-4bad-8d9c-1e7ebaaf331c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:50.260367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557a145-fb1c-4322-8a26-8f390139761e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:50.278990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:50.302782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be69bf-3c59-445e-a52a-edcece35814f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:50.303555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be69bf-3c59-445e-a52a-edcece35814f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.303834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be69bf-3c59-445e-a52a-edcece35814f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.304105 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be69bf-3c59-445e-a52a-edcece35814f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.304756 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be69bf-3c59-445e-a52a-edcece35814f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:50.306135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be69bf-3c59-445e-a52a-edcece35814f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-63be69bf-3c59-445e-a52a-edcece35814f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:50.319817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:50.320795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:50.326628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 473/940] 10.222.0.75 () {56 vars in 1132 bytes} [Thu Jul 14 10:54:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:50.328494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be69bf-3c59-445e-a52a-edcece35814f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:50.356179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08231ee-d47a-4bad-8d9c-1e7ebaaf331c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:50.394126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1a079-7df8-4f13-96b3-ab5a18e4f1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:50.394933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1a079-7df8-4f13-96b3-ab5a18e4f1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.395275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1a079-7df8-4f13-96b3-ab5a18e4f1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.395674 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1a079-7df8-4f13-96b3-ab5a18e4f1c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.396559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1a079-7df8-4f13-96b3-ab5a18e4f1c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:50.396927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1a079-7df8-4f13-96b3-ab5a18e4f1c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:50.398463 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1a079-7df8-4f13-96b3-ab5a18e4f1c9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:50.411322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 468/941] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:50.412106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1a079-7df8-4f13-96b3-ab5a18e4f1c9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:50.438547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be69bf-3c59-445e-a52a-edcece35814f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:50.479878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96664e72-c0f4-4445-8ac6-56164227f7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:50.480702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96664e72-c0f4-4445-8ac6-56164227f7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.481063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96664e72-c0f4-4445-8ac6-56164227f7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.481347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96664e72-c0f4-4445-8ac6-56164227f7c0 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.482073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96664e72-c0f4-4445-8ac6-56164227f7c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:50.482372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96664e72-c0f4-4445-8ac6-56164227f7c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:50.483783 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96664e72-c0f4-4445-8ac6-56164227f7c0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:50.489684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 474/942] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Jul 14 10:54:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:50.491300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96664e72-c0f4-4445-8ac6-56164227f7c0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:50.508159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:50.533321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:50.541055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1a079-7df8-4f13-96b3-ab5a18e4f1c9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:50.582035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f8267-f865-401d-a55b-96d033b43d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:50.582733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f8267-f865-401d-a55b-96d033b43d55 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.583077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f8267-f865-401d-a55b-96d033b43d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.583438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f8267-f865-401d-a55b-96d033b43d55 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.584206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f8267-f865-401d-a55b-96d033b43d55 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:50.584575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f8267-f865-401d-a55b-96d033b43d55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:50.586015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f8267-f865-401d-a55b-96d033b43d55 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:50.596134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 469/943] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Jul 14 10:54:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:50.596134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f8267-f865-401d-a55b-96d033b43d55 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:50.620159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96664e72-c0f4-4445-8ac6-56164227f7c0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:50.661494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229cfce-b711-49b5-972c-da9040399473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:50.662552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229cfce-b711-49b5-972c-da9040399473 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.662929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229cfce-b711-49b5-972c-da9040399473 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.663304 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229cfce-b711-49b5-972c-da9040399473 None admin] PATH_INFO: `/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/f40c0fcc93cf4bbf8d35c1de70d087ba/roles/4d8a62dacbc5492b9fa4b11c57673134` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.665832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229cfce-b711-49b5-972c-da9040399473 None admin] RBAC: Authorizing `identity:create_grant(project_id=a68601f16d1c49548b6e3af0b9819ba3, user_id=f40c0fcc93cf4bbf8d35c1de70d087ba, role_id=4d8a62dacbc5492b9fa4b11c57673134)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:50.667268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229cfce-b711-49b5-972c-da9040399473 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:50.682178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 475/944] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:54:50 2022] PUT /identity/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/f40c0fcc93cf4bbf8d35c1de70d087ba/roles/4d8a62dacbc5492b9fa4b11c57673134 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:54:50.683272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229cfce-b711-49b5-972c-da9040399473 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:50.713708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f8267-f865-401d-a55b-96d033b43d55 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:50.764991 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5734ee-31e6-442a-a318-fa7484d1cf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:50.766235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5734ee-31e6-442a-a318-fa7484d1cf57 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.767150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5734ee-31e6-442a-a318-fa7484d1cf57 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.767924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5734ee-31e6-442a-a318-fa7484d1cf57 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.768848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5734ee-31e6-442a-a318-fa7484d1cf57 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:50.769317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5734ee-31e6-442a-a318-fa7484d1cf57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:50.771260 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5734ee-31e6-442a-a318-fa7484d1cf57 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:50.775686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:50.783941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 470/945] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Jul 14 10:54:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:50.786349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5734ee-31e6-442a-a318-fa7484d1cf57 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:50.820550 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229cfce-b711-49b5-972c-da9040399473 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:50.826193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:50.867414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ebc35-f93e-42dc-af54-354da13c5db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:50.868330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ebc35-f93e-42dc-af54-354da13c5db1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.868787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ebc35-f93e-42dc-af54-354da13c5db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.869212 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ebc35-f93e-42dc-af54-354da13c5db1 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.870220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ebc35-f93e-42dc-af54-354da13c5db1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:50.870735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ebc35-f93e-42dc-af54-354da13c5db1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:50.872429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ebc35-f93e-42dc-af54-354da13c5db1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:50.878489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 476/946] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Jul 14 10:54:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:50.883853 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ebc35-f93e-42dc-af54-354da13c5db1 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:50.923517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5734ee-31e6-442a-a318-fa7484d1cf57 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:50.962116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cdc0f-8f5a-4590-aa0f-06786eed6717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:50.962788 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cdc0f-8f5a-4590-aa0f-06786eed6717 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.963074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cdc0f-8f5a-4590-aa0f-06786eed6717 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.963353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cdc0f-8f5a-4590-aa0f-06786eed6717 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.964180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cdc0f-8f5a-4590-aa0f-06786eed6717 None admin] RBAC: Adding query filter params (interface=public, service_id=57262ba8a68a4bec97d509f14ea94808, region_id=RegionOne) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:50.964496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cdc0f-8f5a-4590-aa0f-06786eed6717 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:50.965907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cdc0f-8f5a-4590-aa0f-06786eed6717 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:50.972009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 471/947] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Jul 14 10:54:50 2022] GET /identity/v3/endpoints?service_id=57262ba8a68a4bec97d509f14ea94808&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:50.977448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cdc0f-8f5a-4590-aa0f-06786eed6717 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:51.012552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ebc35-f93e-42dc-af54-354da13c5db1 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:51.051938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e305ee4-48da-4ec6-9a24-d0da7cc353ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:51.052726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e305ee4-48da-4ec6-9a24-d0da7cc353ef None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:51.053192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e305ee4-48da-4ec6-9a24-d0da7cc353ef None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:51.053544 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e305ee4-48da-4ec6-9a24-d0da7cc353ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:51.054552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e305ee4-48da-4ec6-9a24-d0da7cc353ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:51.056050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e305ee4-48da-4ec6-9a24-d0da7cc353ef None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:51.060614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:51.087154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 477/948] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Jul 14 10:54:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:54:51.094686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:51.118943 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cdc0f-8f5a-4590-aa0f-06786eed6717 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:51.170634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c459bf-5d92-4b7c-b27c-28332835d4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:51.171929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c459bf-5d92-4b7c-b27c-28332835d4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:51.172465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c459bf-5d92-4b7c-b27c-28332835d4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:51.172971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c459bf-5d92-4b7c-b27c-28332835d4e4 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:51.173886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c459bf-5d92-4b7c-b27c-28332835d4e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:51.175568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c459bf-5d92-4b7c-b27c-28332835d4e4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:51.184269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 472/949] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Jul 14 10:54:50 2022] GET /identity/v3/services => generated 1497 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:54:52.096719 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7166fb4-75dc-4903-9f11-e94135c5c51e None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:52.097735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7166fb4-75dc-4903-9f11-e94135c5c51e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:52.098247 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7166fb4-75dc-4903-9f11-e94135c5c51e None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:52.100155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 478/950] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:52.105741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc79b54-21a2-49fa-b985-47f4c82e7b09 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:52.106303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc79b54-21a2-49fa-b985-47f4c82e7b09 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:52.106783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc79b54-21a2-49fa-b985-47f4c82e7b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:52.126628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6dc79b54-21a2-49fa-b985-47f4c82e7b09 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:52.128247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6dc79b54-21a2-49fa-b985-47f4c82e7b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:52.215505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 473/951] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:52.235933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7166fb4-75dc-4903-9f11-e94135c5c51e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:52.352856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7166fb4-75dc-4903-9f11-e94135c5c51e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:52.396046 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09faf91-5583-4073-950e-7e0e267239e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:52.396735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09faf91-5583-4073-950e-7e0e267239e1 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:52.397021 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09faf91-5583-4073-950e-7e0e267239e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:52.397302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09faf91-5583-4073-950e-7e0e267239e1 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:52.398011 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09faf91-5583-4073-950e-7e0e267239e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:52.399500 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09faf91-5583-4073-950e-7e0e267239e1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:52.414396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.sql.core [None req-b09faf91-5583-4073-950e-7e0e267239e1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 10:54:52.414396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 10:54:52.414396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [parameters: {'user_id': '0702ed03d53349cc8fd8b5bfa7b46fdf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 10:54:52.414396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80183) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-b09faf91-5583-4073-950e-7e0e267239e1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result.read() Jul 14 10:54:52.422305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:54:52.423228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application yield resource Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise self.value Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application session.commit() Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 10:54:52.424318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.session.flush() Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application compat.raise_( Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 10:54:52.426268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:54:52.427165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application result.read() Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:54:52.429238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [parameters: {'user_id': '0702ed03d53349cc8fd8b5bfa7b46fdf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:52.430721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 14 10:54:52.432285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 14 10:54:52.433752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 14 10:54:52.433752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 14 10:54:52.433752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 10:54:52.433752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 10:54:52.433752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 10:54:52.433752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:52.433752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 479/952] 10.222.0.75 () {60 vars in 1131 bytes} [Thu Jul 14 10:54:52 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:52.434260 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc79b54-21a2-49fa-b985-47f4c82e7b09 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:52.536626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:52.567726 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc79b54-21a2-49fa-b985-47f4c82e7b09 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:52.606078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9628673-a628-4942-9289-a5ad0565c550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:52.606801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9628673-a628-4942-9289-a5ad0565c550 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:52.607085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9628673-a628-4942-9289-a5ad0565c550 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:52.607358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9628673-a628-4942-9289-a5ad0565c550 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:52.615106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9628673-a628-4942-9289-a5ad0565c550 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:52.616610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9628673-a628-4942-9289-a5ad0565c550 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-e9628673-a628-4942-9289-a5ad0565c550 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:52.631693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:52.633938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:52.635820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:52.635820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:52.635820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:52.635820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:52.635820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:52.635820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:52.635820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:52.635820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:52.635820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:54:52.635820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:52.635820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 474/953] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:54:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:52.645317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09faf91-5583-4073-950e-7e0e267239e1 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:52.702847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:52.758300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09faf91-5583-4073-950e-7e0e267239e1 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:52.796012 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a9592-dab7-498b-9122-0964aff752cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:52.796710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a9592-dab7-498b-9122-0964aff752cf None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:52.796995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a9592-dab7-498b-9122-0964aff752cf None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:52.797281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a9592-dab7-498b-9122-0964aff752cf None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:52.798099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a9592-dab7-498b-9122-0964aff752cf None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:52.798406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a9592-dab7-498b-9122-0964aff752cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:52.800088 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a9592-dab7-498b-9122-0964aff752cf None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:52.841677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 480/954] 10.222.0.75 () {56 vars in 1101 bytes} [Thu Jul 14 10:54:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:52.901803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad6c18-992a-4a4a-bd9d-a06db5d68b0b None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:52.902117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad6c18-992a-4a4a-bd9d-a06db5d68b0b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:52.902528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad6c18-992a-4a4a-bd9d-a06db5d68b0b None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:52.904284 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 475/955] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:52.912950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b72e9-15f1-40be-adff-008b589a2b14 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:52.913394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b72e9-15f1-40be-adff-008b589a2b14 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:52.913764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b72e9-15f1-40be-adff-008b589a2b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:52.933946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a03b72e9-15f1-40be-adff-008b589a2b14 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:52.935575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a03b72e9-15f1-40be-adff-008b589a2b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:53.035672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 481/956] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:53.059317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad6c18-992a-4a4a-bd9d-a06db5d68b0b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:53.180247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad6c18-992a-4a4a-bd9d-a06db5d68b0b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:53.192130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a8f2d-108b-4fc0-bf5d-08ee6954d411 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.192670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a8f2d-108b-4fc0-bf5d-08ee6954d411 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.192670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a8f2d-108b-4fc0-bf5d-08ee6954d411 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.194355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 482/957] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:53.199657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63dfa8-a57a-44dd-8935-11ab76364eaf None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.200462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63dfa8-a57a-44dd-8935-11ab76364eaf None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.200462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63dfa8-a57a-44dd-8935-11ab76364eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.218352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9e63dfa8-a57a-44dd-8935-11ab76364eaf None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:53.219625 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9e63dfa8-a57a-44dd-8935-11ab76364eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:53.228656 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca00438-2be9-4c14-a044-44a4241c02dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:53.229578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca00438-2be9-4c14-a044-44a4241c02dc None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.229767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca00438-2be9-4c14-a044-44a4241c02dc None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.230045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca00438-2be9-4c14-a044-44a4241c02dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.240621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:53.246485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca00438-2be9-4c14-a044-44a4241c02dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-dca00438-2be9-4c14-a044-44a4241c02dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:54:53.256359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:53.257360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:53.258190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 476/958] 10.222.0.75 () {56 vars in 1120 bytes} [Thu Jul 14 10:54:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:53.263772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca00438-2be9-4c14-a044-44a4241c02dc None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:53.269129 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:53.314517 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 483/959] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:54:53.333286 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ef90b-cb40-4f31-9964-e3f04dc88d63 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.333748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ef90b-cb40-4f31-9964-e3f04dc88d63 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.334028 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ef90b-cb40-4f31-9964-e3f04dc88d63 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.335895 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 484/960] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:54:53.337746 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ef90b-cb40-4f31-9964-e3f04dc88d63 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:53.374291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca00438-2be9-4c14-a044-44a4241c02dc None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:53.408372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffda0f-d629-4b9d-b803-8d78fe45958b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:53.409054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffda0f-d629-4b9d-b803-8d78fe45958b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.409343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffda0f-d629-4b9d-b803-8d78fe45958b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.409626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffda0f-d629-4b9d-b803-8d78fe45958b None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.410407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffda0f-d629-4b9d-b803-8d78fe45958b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:53.410717 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffda0f-d629-4b9d-b803-8d78fe45958b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:53.412139 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffda0f-d629-4b9d-b803-8d78fe45958b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:53.427161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 477/961] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Jul 14 10:54:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:53.427161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ef53a-244c-4913-b29c-5039160413f3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.427161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ef53a-244c-4913-b29c-5039160413f3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.427161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ef53a-244c-4913-b29c-5039160413f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.440756 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ef90b-cb40-4f31-9964-e3f04dc88d63 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:53.443922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-3e0ef53a-244c-4913-b29c-5039160413f3 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:53.445131 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3e0ef53a-244c-4913-b29c-5039160413f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:53.475403 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957adfe-6d66-4d4a-9a24-544cf565b069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:53.476227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957adfe-6d66-4d4a-9a24-544cf565b069 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.476513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957adfe-6d66-4d4a-9a24-544cf565b069 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.476789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957adfe-6d66-4d4a-9a24-544cf565b069 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.477438 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957adfe-6d66-4d4a-9a24-544cf565b069 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:53.478811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957adfe-6d66-4d4a-9a24-544cf565b069 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-7957adfe-6d66-4d4a-9a24-544cf565b069 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:53.490928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:53.492061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:53.493112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 485/962] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:53.494489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957adfe-6d66-4d4a-9a24-544cf565b069 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:53.504592 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:53.515927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 478/963] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:53.516831 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ef53a-244c-4913-b29c-5039160413f3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:53.534721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:53.605525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957adfe-6d66-4d4a-9a24-544cf565b069 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:53.624713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ef53a-244c-4913-b29c-5039160413f3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:53.639676 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1b4a2-2f95-482d-9672-2c1da535371a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:53.640484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1b4a2-2f95-482d-9672-2c1da535371a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.640768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1b4a2-2f95-482d-9672-2c1da535371a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.641051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1b4a2-2f95-482d-9672-2c1da535371a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.652149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1b4a2-2f95-482d-9672-2c1da535371a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:53.653599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1b4a2-2f95-482d-9672-2c1da535371a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:53.658490 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502125c-95ed-479c-881b-dcdf059f799c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:53.659156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502125c-95ed-479c-881b-dcdf059f799c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.659468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502125c-95ed-479c-881b-dcdf059f799c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.659745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502125c-95ed-479c-881b-dcdf059f799c None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.660445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502125c-95ed-479c-881b-dcdf059f799c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:53.660749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502125c-95ed-479c-881b-dcdf059f799c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:53.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502125c-95ed-479c-881b-dcdf059f799c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:53.666753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 479/964] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:54:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:53.667548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502125c-95ed-479c-881b-dcdf059f799c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-42f1b4a2-2f95-482d-9672-2c1da535371a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:53.670904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:53.671921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:53.672819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:53.672819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:53.672819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:53.672819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:53.672819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 486/965] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:53.673185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1b4a2-2f95-482d-9672-2c1da535371a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:53.723358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:53.758735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:53.768433 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502125c-95ed-479c-881b-dcdf059f799c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:53.774421 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1b4a2-2f95-482d-9672-2c1da535371a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:53.801514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530cc11-0b63-4fed-adc1-7aa06f6b6d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:53.802316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530cc11-0b63-4fed-adc1-7aa06f6b6d2c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.802597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530cc11-0b63-4fed-adc1-7aa06f6b6d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.802873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530cc11-0b63-4fed-adc1-7aa06f6b6d2c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.803554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530cc11-0b63-4fed-adc1-7aa06f6b6d2c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:53.804926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530cc11-0b63-4fed-adc1-7aa06f6b6d2c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:53.806571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a155ad-0139-4877-bd55-77a827653cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:53.807234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a155ad-0139-4877-bd55-77a827653cbe None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.807548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a155ad-0139-4877-bd55-77a827653cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.807829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a155ad-0139-4877-bd55-77a827653cbe None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.808793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a155ad-0139-4877-bd55-77a827653cbe None admin] RBAC: Adding query filter params (type=compute) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:53.809099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a155ad-0139-4877-bd55-77a827653cbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:53.810443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a155ad-0139-4877-bd55-77a827653cbe None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:53.814873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 487/966] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:54:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:53.815658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a155ad-0139-4877-bd55-77a827653cbe None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-e530cc11-0b63-4fed-adc1-7aa06f6b6d2c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:53.817616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:53.818581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:53.819501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 480/967] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Jul 14 10:54:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:53.824998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530cc11-0b63-4fed-adc1-7aa06f6b6d2c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:53.936183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a155ad-0139-4877-bd55-77a827653cbe None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:53.950088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530cc11-0b63-4fed-adc1-7aa06f6b6d2c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:53.970079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:53.976118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94354f-77e8-4ed5-bb6c-a19c5800d19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:53.976766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94354f-77e8-4ed5-bb6c-a19c5800d19a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.977057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94354f-77e8-4ed5-bb6c-a19c5800d19a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.977280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94354f-77e8-4ed5-bb6c-a19c5800d19a None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.978018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94354f-77e8-4ed5-bb6c-a19c5800d19a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:53.978319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94354f-77e8-4ed5-bb6c-a19c5800d19a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:53.980163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94354f-77e8-4ed5-bb6c-a19c5800d19a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:53.993283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 488/968] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:53.995503 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f5eb0-3558-480d-8b97-ff4709ec0461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:53.996181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f5eb0-3558-480d-8b97-ff4709ec0461 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:53.996468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f5eb0-3558-480d-8b97-ff4709ec0461 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:53.996744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f5eb0-3558-480d-8b97-ff4709ec0461 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:53.997456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f5eb0-3558-480d-8b97-ff4709ec0461 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:53.997763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f5eb0-3558-480d-8b97-ff4709ec0461 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:53.998794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94354f-77e8-4ed5-bb6c-a19c5800d19a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:53.999155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f5eb0-3558-480d-8b97-ff4709ec0461 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:54.005031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 481/969] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Jul 14 10:54:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:54.011329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f5eb0-3558-480d-8b97-ff4709ec0461 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:54.013408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:54.113860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94354f-77e8-4ed5-bb6c-a19c5800d19a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:54.120492 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f5eb0-3558-480d-8b97-ff4709ec0461 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:54.151463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998b34a-2a49-4df9-8f5e-71ef83f25c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:54.152298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998b34a-2a49-4df9-8f5e-71ef83f25c16 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:54.152590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998b34a-2a49-4df9-8f5e-71ef83f25c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:54.152874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998b34a-2a49-4df9-8f5e-71ef83f25c16 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:54.155785 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaa1e5-d128-4ff8-8743-fa1a0c9866fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:54.156465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaa1e5-d128-4ff8-8743-fa1a0c9866fd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:54.156744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaa1e5-d128-4ff8-8743-fa1a0c9866fd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:54.157018 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaa1e5-d128-4ff8-8743-fa1a0c9866fd None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:54.157725 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdaa1e5-d128-4ff8-8743-fa1a0c9866fd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:54.158034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdaa1e5-d128-4ff8-8743-fa1a0c9866fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:54.159440 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdaa1e5-d128-4ff8-8743-fa1a0c9866fd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:54.162919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998b34a-2a49-4df9-8f5e-71ef83f25c16 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:54.164619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998b34a-2a49-4df9-8f5e-71ef83f25c16 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:54.164936 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 482/970] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Jul 14 10:54:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:54.171052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaa1e5-d128-4ff8-8743-fa1a0c9866fd None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a998b34a-2a49-4df9-8f5e-71ef83f25c16 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:54:54.185309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:54.186323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:54.187283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:54.187283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:54.187283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:54:54.187283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:54:54.187283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:54:54.187283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:54:54.187283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:54:54.187283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:54:54.187283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:54:54.187283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:54.187283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 489/971] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Jul 14 10:54:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:54.193976 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998b34a-2a49-4df9-8f5e-71ef83f25c16 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:54.231111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:54.273015 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:54.295969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaa1e5-d128-4ff8-8743-fa1a0c9866fd None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:54.310577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998b34a-2a49-4df9-8f5e-71ef83f25c16 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:54.332992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae5570-5eab-4fbe-b45c-94dc6256c298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:54.333665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae5570-5eab-4fbe-b45c-94dc6256c298 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:54.333951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae5570-5eab-4fbe-b45c-94dc6256c298 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:54.334233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae5570-5eab-4fbe-b45c-94dc6256c298 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:54.334975 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae5570-5eab-4fbe-b45c-94dc6256c298 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:54.336400 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae5570-5eab-4fbe-b45c-94dc6256c298 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:54.350081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816dc77-1d4f-4306-b61c-40df8c03d913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:54.350822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816dc77-1d4f-4306-b61c-40df8c03d913 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:54.351031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816dc77-1d4f-4306-b61c-40df8c03d913 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:54.351536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816dc77-1d4f-4306-b61c-40df8c03d913 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:54.352200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816dc77-1d4f-4306-b61c-40df8c03d913 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:54.352510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816dc77-1d4f-4306-b61c-40df8c03d913 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:54.353963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816dc77-1d4f-4306-b61c-40df8c03d913 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:54.367662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 483/972] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Jul 14 10:54:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:54:54.374214 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 490/973] 10.222.0.75 () {56 vars in 1149 bytes} [Thu Jul 14 10:54:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:54.380891 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae5570-5eab-4fbe-b45c-94dc6256c298 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:54.495150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:54.502694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae5570-5eab-4fbe-b45c-94dc6256c298 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:54.541718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f586a-7e1e-4d93-8520-fdd17291abcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:54.542375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f586a-7e1e-4d93-8520-fdd17291abcb None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:54.542657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f586a-7e1e-4d93-8520-fdd17291abcb None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:54.542935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f586a-7e1e-4d93-8520-fdd17291abcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:54.617847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f586a-7e1e-4d93-8520-fdd17291abcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:54.619319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f586a-7e1e-4d93-8520-fdd17291abcb None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:54.717524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:54.728285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 484/974] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:54:54 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:54:54.729744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816dc77-1d4f-4306-b61c-40df8c03d913 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:54.834682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816dc77-1d4f-4306-b61c-40df8c03d913 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:54.862889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:54.871296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7bebe-cf38-404d-b790-af4753d72b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:54.872123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7bebe-cf38-404d-b790-af4753d72b51 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:54.872405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7bebe-cf38-404d-b790-af4753d72b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:54.872680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7bebe-cf38-404d-b790-af4753d72b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:54.883314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7bebe-cf38-404d-b790-af4753d72b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:54.884838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7bebe-cf38-404d-b790-af4753d72b51 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-36c7bebe-cf38-404d-b790-af4753d72b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:54:54.903163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:54.904310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:54.909894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:54:54.909894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:54:54.909894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:54:54.909894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:54.909894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 491/975] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:54.912502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f586a-7e1e-4d93-8520-fdd17291abcb None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:55.021356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f586a-7e1e-4d93-8520-fdd17291abcb None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:55.059853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4734d-4b40-4a5c-989f-79e5617bf855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:55.060517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4734d-4b40-4a5c-989f-79e5617bf855 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:55.060809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4734d-4b40-4a5c-989f-79e5617bf855 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:55.061083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4734d-4b40-4a5c-989f-79e5617bf855 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:55.061810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4734d-4b40-4a5c-989f-79e5617bf855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:55.062122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4734d-4b40-4a5c-989f-79e5617bf855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:55.063559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4734d-4b40-4a5c-989f-79e5617bf855 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:55.079806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 485/976] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:55.081623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7bebe-cf38-404d-b790-af4753d72b51 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:55.183602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7bebe-cf38-404d-b790-af4753d72b51 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:55.226009 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ea145-b217-4026-947d-b5688075811e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:55.226809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ea145-b217-4026-947d-b5688075811e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:55.227095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ea145-b217-4026-947d-b5688075811e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:55.227374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ea145-b217-4026-947d-b5688075811e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:55.234294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ea145-b217-4026-947d-b5688075811e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:55.235775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ea145-b217-4026-947d-b5688075811e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6f1ea145-b217-4026-947d-b5688075811e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:54:55.249467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:55.250319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:55.251209 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:54:55.251209 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:54:55.251209 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:54:55.251209 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:54:55.251209 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:54:55.251209 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:54:55.251209 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:54:55.251209 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:55.251209 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 492/977] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:54:55.257101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4734d-4b40-4a5c-989f-79e5617bf855 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:55.315613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:55.358058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4734d-4b40-4a5c-989f-79e5617bf855 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:55.391227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f10f8-ce4c-449f-ad21-57e5a028ec8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:55.391912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f10f8-ce4c-449f-ad21-57e5a028ec8f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:55.392193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f10f8-ce4c-449f-ad21-57e5a028ec8f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:55.392469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f10f8-ce4c-449f-ad21-57e5a028ec8f None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:55.393242 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f10f8-ce4c-449f-ad21-57e5a028ec8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:55.393547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f10f8-ce4c-449f-ad21-57e5a028ec8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:55.395562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f10f8-ce4c-449f-ad21-57e5a028ec8f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:55.412276 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 486/978] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:54:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:55.414067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ea145-b217-4026-947d-b5688075811e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:55.458182 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:55.520128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ea145-b217-4026-947d-b5688075811e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:55.555947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c1f20-9547-494f-af02-048fb1191b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:55.556863 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c1f20-9547-494f-af02-048fb1191b72 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:55.557194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c1f20-9547-494f-af02-048fb1191b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:55.557415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c1f20-9547-494f-af02-048fb1191b72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:55.558569 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c1f20-9547-494f-af02-048fb1191b72 None admin] RBAC: Adding query filter params (role.id=4d8a62dacbc5492b9fa4b11c57673134, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=f40c0fcc93cf4bbf8d35c1de70d087ba) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:55.558882 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c1f20-9547-494f-af02-048fb1191b72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:55.560395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c1f20-9547-494f-af02-048fb1191b72 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:55.575428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 493/979] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:54:55 2022] GET /identity/v3/role_assignments?user.id=f40c0fcc93cf4bbf8d35c1de70d087ba&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=4d8a62dacbc5492b9fa4b11c57673134 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:55.879876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef831128-bb4a-4efa-ba66-7e521389bd97 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:55.880265 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef831128-bb4a-4efa-ba66-7e521389bd97 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:55.880370 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef831128-bb4a-4efa-ba66-7e521389bd97 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:55.881909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 487/980] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:55.887145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344a947-3276-4cec-b5ec-5feb308fb075 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:55.887473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344a947-3276-4cec-b5ec-5feb308fb075 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:55.887763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344a947-3276-4cec-b5ec-5feb308fb075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:55.904730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0344a947-3276-4cec-b5ec-5feb308fb075 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:55.905927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0344a947-3276-4cec-b5ec-5feb308fb075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:55.973958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 494/981] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:55.996296 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef831128-bb4a-4efa-ba66-7e521389bd97 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:56.100958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef831128-bb4a-4efa-ba66-7e521389bd97 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:56.119277 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:56.137516 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62dfb9-7dbc-4d03-8d9e-db16e5f745b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:56.138113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62dfb9-7dbc-4d03-8d9e-db16e5f745b1 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:56.138399 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62dfb9-7dbc-4d03-8d9e-db16e5f745b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:56.138678 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62dfb9-7dbc-4d03-8d9e-db16e5f745b1 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:56.139407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62dfb9-7dbc-4d03-8d9e-db16e5f745b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:56.140855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62dfb9-7dbc-4d03-8d9e-db16e5f745b1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:56.159412 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 488/982] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:54:55 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:54:56.449383 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4381f-3b8f-4897-9d1e-a2dc725d6864 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:56.449951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4381f-3b8f-4897-9d1e-a2dc725d6864 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:56.449951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4381f-3b8f-4897-9d1e-a2dc725d6864 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:56.451407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 495/983] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:56.457930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb19742-e778-4586-8148-22dda60d5f38 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:56.458431 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb19742-e778-4586-8148-22dda60d5f38 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:56.458504 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb19742-e778-4586-8148-22dda60d5f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:56.476525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9eb19742-e778-4586-8148-22dda60d5f38 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:56.477751 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9eb19742-e778-4586-8148-22dda60d5f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:56.558849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 489/984] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:56.582574 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4381f-3b8f-4897-9d1e-a2dc725d6864 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:56.622549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:56.699027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4381f-3b8f-4897-9d1e-a2dc725d6864 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:56.737196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7d9fb-8bc1-43b7-b951-f5d7a58e7fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:56.737931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7d9fb-8bc1-43b7-b951-f5d7a58e7fac None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:56.738248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7d9fb-8bc1-43b7-b951-f5d7a58e7fac None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:56.738525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7d9fb-8bc1-43b7-b951-f5d7a58e7fac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:56.739181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e7d9fb-8bc1-43b7-b951-f5d7a58e7fac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:56.740639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e7d9fb-8bc1-43b7-b951-f5d7a58e7fac None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a7e7d9fb-8bc1-43b7-b951-f5d7a58e7fac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:56.753412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:56.754971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:56.756463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 496/985] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Jul 14 10:54:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:56.761156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb19742-e778-4586-8148-22dda60d5f38 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:56.861783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb19742-e778-4586-8148-22dda60d5f38 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:56.864170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:56.899311 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313fe8b-f60b-49d5-bbb6-4be32f869ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:56.900008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313fe8b-f60b-49d5-bbb6-4be32f869ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:56.900289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313fe8b-f60b-49d5-bbb6-4be32f869ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:56.900563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313fe8b-f60b-49d5-bbb6-4be32f869ec9 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:56.901262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313fe8b-f60b-49d5-bbb6-4be32f869ec9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:56.901566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313fe8b-f60b-49d5-bbb6-4be32f869ec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:56.902931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313fe8b-f60b-49d5-bbb6-4be32f869ec9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:56.908084 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 490/986] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Jul 14 10:54:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:56.913587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7d9fb-8bc1-43b7-b951-f5d7a58e7fac None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:57.012316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:57.016879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7d9fb-8bc1-43b7-b951-f5d7a58e7fac None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:57.052747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013b53c-9f56-41fb-93c4-742423d767bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:57.053419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013b53c-9f56-41fb-93c4-742423d767bd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:57.053703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013b53c-9f56-41fb-93c4-742423d767bd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:57.053984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013b53c-9f56-41fb-93c4-742423d767bd None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:57.054689 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013b53c-9f56-41fb-93c4-742423d767bd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:57.054997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013b53c-9f56-41fb-93c4-742423d767bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:57.056399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013b53c-9f56-41fb-93c4-742423d767bd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:57.067435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 497/987] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Jul 14 10:54:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:57.068645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313fe8b-f60b-49d5-bbb6-4be32f869ec9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:57.169992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313fe8b-f60b-49d5-bbb6-4be32f869ec9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:57.216151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ced01-d619-4517-a7bb-133b5c11db44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:57.216822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ced01-d619-4517-a7bb-133b5c11db44 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:57.217103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ced01-d619-4517-a7bb-133b5c11db44 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:57.217375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ced01-d619-4517-a7bb-133b5c11db44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:57.218170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ced01-d619-4517-a7bb-133b5c11db44 None admin] RBAC: Adding query filter params (interface=admin, service_id=57262ba8a68a4bec97d509f14ea94808, region_id=RegionOne) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:57.218478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ced01-d619-4517-a7bb-133b5c11db44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:57.219892 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ced01-d619-4517-a7bb-133b5c11db44 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:57.229828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 491/988] 10.222.0.75 () {56 vars in 1249 bytes} [Thu Jul 14 10:54:57 2022] GET /identity/v3/endpoints?service_id=57262ba8a68a4bec97d509f14ea94808&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:57.231266 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013b53c-9f56-41fb-93c4-742423d767bd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:57.339290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013b53c-9f56-41fb-93c4-742423d767bd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:57.373670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cb210-2f6c-4455-ae30-369664031860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:57.374345 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cb210-2f6c-4455-ae30-369664031860 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:57.374632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cb210-2f6c-4455-ae30-369664031860 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:57.374909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cb210-2f6c-4455-ae30-369664031860 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:57.375631 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cb210-2f6c-4455-ae30-369664031860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:57.376994 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cb210-2f6c-4455-ae30-369664031860 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:57.391414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 498/989] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Jul 14 10:54:57 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:54:57.671338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2057a29-9da7-4985-9df2-5715ee80d79e None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:57.671882 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2057a29-9da7-4985-9df2-5715ee80d79e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:57.671959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2057a29-9da7-4985-9df2-5715ee80d79e None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:57.673404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 492/990] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:57.679304 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9829ca3-19b9-4739-8bf0-8ac5a4c7a78f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:57.679749 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9829ca3-19b9-4739-8bf0-8ac5a4c7a78f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:57.679935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9829ca3-19b9-4739-8bf0-8ac5a4c7a78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:57.696888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c9829ca3-19b9-4739-8bf0-8ac5a4c7a78f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:57.698071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c9829ca3-19b9-4739-8bf0-8ac5a4c7a78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:57.718713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:57.768497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 499/991] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:57.790929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2057a29-9da7-4985-9df2-5715ee80d79e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:57.823227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:57.899296 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2057a29-9da7-4985-9df2-5715ee80d79e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:57.934795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cd5eb-ce15-49b7-90e5-76b52e8bd92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:57.935528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cd5eb-ce15-49b7-90e5-76b52e8bd92e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:57.935812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cd5eb-ce15-49b7-90e5-76b52e8bd92e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:57.936085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cd5eb-ce15-49b7-90e5-76b52e8bd92e None admin] PATH_INFO: `/v3/services/image` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:57.936729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76cd5eb-ce15-49b7-90e5-76b52e8bd92e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:57.938099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76cd5eb-ce15-49b7-90e5-76b52e8bd92e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-c76cd5eb-ce15-49b7-90e5-76b52e8bd92e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:57.949573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:57.950542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:57.951410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 493/992] 10.222.0.75 () {56 vars in 1105 bytes} [Thu Jul 14 10:54:57 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:57.957330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9829ca3-19b9-4739-8bf0-8ac5a4c7a78f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:58.060909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9829ca3-19b9-4739-8bf0-8ac5a4c7a78f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:58.094605 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64ef13-6f39-4714-879d-dbeb1621c59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:58.095274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64ef13-6f39-4714-879d-dbeb1621c59d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:58.095580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64ef13-6f39-4714-879d-dbeb1621c59d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:58.095859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64ef13-6f39-4714-879d-dbeb1621c59d None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:58.096562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b64ef13-6f39-4714-879d-dbeb1621c59d None admin] RBAC: Adding query filter params (name=image) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:58.096872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b64ef13-6f39-4714-879d-dbeb1621c59d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:58.098801 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b64ef13-6f39-4714-879d-dbeb1621c59d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:58.108294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 500/993] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:58.109482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cd5eb-ce15-49b7-90e5-76b52e8bd92e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:58.214426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:58.221274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cd5eb-ce15-49b7-90e5-76b52e8bd92e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:58.243875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a3f2c-547b-4445-881d-251e614bdc40 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:58.244177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a3f2c-547b-4445-881d-251e614bdc40 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:58.244449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a3f2c-547b-4445-881d-251e614bdc40 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:58.245830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 501/994] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:54:58.250882 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53141986-3ed5-4966-bf3a-7fc0b2f3df53 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:58.251170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53141986-3ed5-4966-bf3a-7fc0b2f3df53 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:58.251479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53141986-3ed5-4966-bf3a-7fc0b2f3df53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:58.259017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b276444-67e5-4c0c-baa9-78e375714175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:58.259729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b276444-67e5-4c0c-baa9-78e375714175 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:58.260013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b276444-67e5-4c0c-baa9-78e375714175 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:58.260292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b276444-67e5-4c0c-baa9-78e375714175 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:58.260995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b276444-67e5-4c0c-baa9-78e375714175 None admin] RBAC: Adding query filter params (type=image) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:58.261303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b276444-67e5-4c0c-baa9-78e375714175 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:58.262680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b276444-67e5-4c0c-baa9-78e375714175 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:58.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 494/995] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:54:58 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:58.276900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-53141986-3ed5-4966-bf3a-7fc0b2f3df53 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:58.278214 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-53141986-3ed5-4966-bf3a-7fc0b2f3df53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:58.295312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:58.358706 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 502/996] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:54:58.381527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b276444-67e5-4c0c-baa9-78e375714175 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:58.490205 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b276444-67e5-4c0c-baa9-78e375714175 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:58.527809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529934e-cdfc-450e-87e5-c4b2ec17d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:58.528599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529934e-cdfc-450e-87e5-c4b2ec17d295 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:58.528883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529934e-cdfc-450e-87e5-c4b2ec17d295 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:58.529159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529934e-cdfc-450e-87e5-c4b2ec17d295 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:58.529839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529934e-cdfc-450e-87e5-c4b2ec17d295 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:58.531287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529934e-cdfc-450e-87e5-c4b2ec17d295 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-f529934e-cdfc-450e-87e5-c4b2ec17d295 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:58.544640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:58.545722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:54:58.552037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 495/997] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:54:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:58.554498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53141986-3ed5-4966-bf3a-7fc0b2f3df53 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:58.658711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53141986-3ed5-4966-bf3a-7fc0b2f3df53 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:58.694242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f8b88-0fdf-4c81-92fa-307cc1504d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:58.695016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f8b88-0fdf-4c81-92fa-307cc1504d43 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:58.695202 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f8b88-0fdf-4c81-92fa-307cc1504d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:58.695512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f8b88-0fdf-4c81-92fa-307cc1504d43 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:58.696249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f8b88-0fdf-4c81-92fa-307cc1504d43 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:58.696580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f8b88-0fdf-4c81-92fa-307cc1504d43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:58.698308 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f8b88-0fdf-4c81-92fa-307cc1504d43 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:58.708517 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 503/998] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:54:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:58.710138 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529934e-cdfc-450e-87e5-c4b2ec17d295 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:58.737677 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:58.820572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529934e-cdfc-450e-87e5-c4b2ec17d295 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:58.856090 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5819d2-2e07-44ef-92bd-1d669f1bed0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:58.856764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5819d2-2e07-44ef-92bd-1d669f1bed0e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:58.857052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5819d2-2e07-44ef-92bd-1d669f1bed0e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:58.857332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5819d2-2e07-44ef-92bd-1d669f1bed0e None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:58.858038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5819d2-2e07-44ef-92bd-1d669f1bed0e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:58.858346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5819d2-2e07-44ef-92bd-1d669f1bed0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:58.859760 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5819d2-2e07-44ef-92bd-1d669f1bed0e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:58.864832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 496/999] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:54:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:54:58.870707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f8b88-0fdf-4c81-92fa-307cc1504d43 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:58.886587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:58.973001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f8b88-0fdf-4c81-92fa-307cc1504d43 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:59.007621 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c973b-1e6c-4047-bfce-6ab3dbd18189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:59.008290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c973b-1e6c-4047-bfce-6ab3dbd18189 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:59.008577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c973b-1e6c-4047-bfce-6ab3dbd18189 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:59.008861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c973b-1e6c-4047-bfce-6ab3dbd18189 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:59.009651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c973b-1e6c-4047-bfce-6ab3dbd18189 None admin] RBAC: Adding query filter params (interface=public, service_id=712ed36a27ed4db2bb762200584b3c91, region_id=RegionOne) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:59.009966 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c973b-1e6c-4047-bfce-6ab3dbd18189 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:59.011326 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c973b-1e6c-4047-bfce-6ab3dbd18189 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:59.016610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 504/1000] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Jul 14 10:54:58 2022] GET /identity/v3/endpoints?service_id=712ed36a27ed4db2bb762200584b3c91&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:59.023182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5819d2-2e07-44ef-92bd-1d669f1bed0e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:59.114926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:59.127974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5819d2-2e07-44ef-92bd-1d669f1bed0e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:59.161485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ef79c-164d-4015-8707-1078e5f0116f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:59.162147 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ef79c-164d-4015-8707-1078e5f0116f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:59.162430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ef79c-164d-4015-8707-1078e5f0116f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:59.162706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ef79c-164d-4015-8707-1078e5f0116f None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:59.163373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ef79c-164d-4015-8707-1078e5f0116f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:59.164763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ef79c-164d-4015-8707-1078e5f0116f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:59.171103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 497/1001] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Jul 14 10:54:59 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:54:59.417719 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fc5cf-d7da-442c-a7f9-6846b439a7a6 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:59.418055 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fc5cf-d7da-442c-a7f9-6846b439a7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:59.418281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fc5cf-d7da-442c-a7f9-6846b439a7a6 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:59.419872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 505/1002] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:59.424940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee3a32-d865-46c5-a7f9-0adc54b2165a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:59.425239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee3a32-d865-46c5-a7f9-0adc54b2165a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:59.426050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee3a32-d865-46c5-a7f9-0adc54b2165a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:59.443270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-54ee3a32-d865-46c5-a7f9-0adc54b2165a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:59.444478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-54ee3a32-d865-46c5-a7f9-0adc54b2165a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:59.513723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 498/1003] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:54:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:54:59.535799 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fc5cf-d7da-442c-a7f9-6846b439a7a6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:59.618326 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:59.641322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fc5cf-d7da-442c-a7f9-6846b439a7a6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:59.675288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b84e48-a117-44c0-a720-d5a6b07c698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:59.676581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b84e48-a117-44c0-a720-d5a6b07c698e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:59.676860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b84e48-a117-44c0-a720-d5a6b07c698e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:59.677154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b84e48-a117-44c0-a720-d5a6b07c698e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:59.677800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b84e48-a117-44c0-a720-d5a6b07c698e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:59.679152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b84e48-a117-44c0-a720-d5a6b07c698e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-57b84e48-a117-44c0-a720-d5a6b07c698e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:54:59.692283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:54:59.693322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:54:59.694164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:54:59.696135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 506/1004] 10.222.0.75 () {56 vars in 1126 bytes} [Thu Jul 14 10:54:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:54:59.701383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee3a32-d865-46c5-a7f9-0adc54b2165a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:59.780581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:59.801193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee3a32-d865-46c5-a7f9-0adc54b2165a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:59.834794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29205821-d027-4786-8196-14b72ef1c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:59.835489 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29205821-d027-4786-8196-14b72ef1c327 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:59.835775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29205821-d027-4786-8196-14b72ef1c327 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:59.836050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29205821-d027-4786-8196-14b72ef1c327 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:59.836747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29205821-d027-4786-8196-14b72ef1c327 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:59.837052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29205821-d027-4786-8196-14b72ef1c327 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:59.838420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29205821-d027-4786-8196-14b72ef1c327 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:59.847803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 499/1005] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Jul 14 10:54:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:59.849772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b84e48-a117-44c0-a720-d5a6b07c698e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:59.952950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b84e48-a117-44c0-a720-d5a6b07c698e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:59.986892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3703c-4fc1-4769-bc2a-453c18b38ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:59.987580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3703c-4fc1-4769-bc2a-453c18b38ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:59.987885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3703c-4fc1-4769-bc2a-453c18b38ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:59.988692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3703c-4fc1-4769-bc2a-453c18b38ac3 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:59.989397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3703c-4fc1-4769-bc2a-453c18b38ac3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:54:59.989703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3703c-4fc1-4769-bc2a-453c18b38ac3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:59.991067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3703c-4fc1-4769-bc2a-453c18b38ac3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:00.000613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 507/1006] 10.222.0.75 () {56 vars in 1122 bytes} [Thu Jul 14 10:54:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:00.001874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29205821-d027-4786-8196-14b72ef1c327 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:00.104912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29205821-d027-4786-8196-14b72ef1c327 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:00.138048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80b0fc-462b-4cf4-867d-fb2fb7a39ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:00.138715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80b0fc-462b-4cf4-867d-fb2fb7a39ddc None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:00.139001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80b0fc-462b-4cf4-867d-fb2fb7a39ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:00.139281 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80b0fc-462b-4cf4-867d-fb2fb7a39ddc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:00.140044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80b0fc-462b-4cf4-867d-fb2fb7a39ddc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:00.141472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80b0fc-462b-4cf4-867d-fb2fb7a39ddc None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:00.159325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 500/1007] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Jul 14 10:55:00 2022] POST /identity/v3/endpoints => generated 325 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:55:00.356128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc1d24-55cb-49ad-8328-a5be0f20a902 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:00.356343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc1d24-55cb-49ad-8328-a5be0f20a902 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:00.356629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc1d24-55cb-49ad-8328-a5be0f20a902 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:00.358006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 508/1008] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:00.363331 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79904737-b517-4e37-9225-814921a5264c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:00.364152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79904737-b517-4e37-9225-814921a5264c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:00.364152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79904737-b517-4e37-9225-814921a5264c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:00.375609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:00.382398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-79904737-b517-4e37-9225-814921a5264c None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:00.384127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-79904737-b517-4e37-9225-814921a5264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:00.457468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 501/1009] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:00 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:00.479789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc1d24-55cb-49ad-8328-a5be0f20a902 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:00.489942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:00.587168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc1d24-55cb-49ad-8328-a5be0f20a902 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:00.621144 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe0691-e4e6-42b1-9839-3e2b5578cc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:00.622033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe0691-e4e6-42b1-9839-3e2b5578cc27 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:00.622481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe0691-e4e6-42b1-9839-3e2b5578cc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:00.622903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe0691-e4e6-42b1-9839-3e2b5578cc27 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:00.623811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe0691-e4e6-42b1-9839-3e2b5578cc27 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:00.625420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe0691-e4e6-42b1-9839-3e2b5578cc27 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:00.641211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 509/1010] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:55:00 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:55:01.257634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629cf15-fba7-49e7-a12b-a59832f8a725 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:01.258146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629cf15-fba7-49e7-a12b-a59832f8a725 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:01.258223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629cf15-fba7-49e7-a12b-a59832f8a725 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:01.259770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 502/1011] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:01.264899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7846a3-27e0-432e-90cb-37a77f37f472 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:01.265345 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7846a3-27e0-432e-90cb-37a77f37f472 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:01.265534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7846a3-27e0-432e-90cb-37a77f37f472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:01.282232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0a7846a3-27e0-432e-90cb-37a77f37f472 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:01.283440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0a7846a3-27e0-432e-90cb-37a77f37f472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:01.346595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:01.362822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 510/1012] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:01 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:01.385816 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629cf15-fba7-49e7-a12b-a59832f8a725 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:01.489464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629cf15-fba7-49e7-a12b-a59832f8a725 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:01.517783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:01.523955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45688193-7e8f-4907-94ba-8a9d56124c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:01.524750 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45688193-7e8f-4907-94ba-8a9d56124c25 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:01.525037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45688193-7e8f-4907-94ba-8a9d56124c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:01.525317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45688193-7e8f-4907-94ba-8a9d56124c25 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:01.525966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45688193-7e8f-4907-94ba-8a9d56124c25 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:01.527337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45688193-7e8f-4907-94ba-8a9d56124c25 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-45688193-7e8f-4907-94ba-8a9d56124c25 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:01.539581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:01.540585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:01.546409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 503/1013] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:55:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:01.548723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7846a3-27e0-432e-90cb-37a77f37f472 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:01.651099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7846a3-27e0-432e-90cb-37a77f37f472 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:01.685470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e61d1-e6b6-485e-80c0-ff7c8323e5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:01.686140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e61d1-e6b6-485e-80c0-ff7c8323e5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:01.686431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e61d1-e6b6-485e-80c0-ff7c8323e5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:01.686711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e61d1-e6b6-485e-80c0-ff7c8323e5e2 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:01.687439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e61d1-e6b6-485e-80c0-ff7c8323e5e2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:01.687753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e61d1-e6b6-485e-80c0-ff7c8323e5e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:01.689122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e61d1-e6b6-485e-80c0-ff7c8323e5e2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:01.698579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 511/1014] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:55:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:01.700191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45688193-7e8f-4907-94ba-8a9d56124c25 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:01.813319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45688193-7e8f-4907-94ba-8a9d56124c25 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:01.847339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d4c2a-35c6-466f-86a6-449fa7b0decd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:01.848545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d4c2a-35c6-466f-86a6-449fa7b0decd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:01.848829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d4c2a-35c6-466f-86a6-449fa7b0decd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:01.849415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d4c2a-35c6-466f-86a6-449fa7b0decd None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:01.850151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d4c2a-35c6-466f-86a6-449fa7b0decd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:01.850579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d4c2a-35c6-466f-86a6-449fa7b0decd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:01.852345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d4c2a-35c6-466f-86a6-449fa7b0decd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:01.862879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 504/1015] 10.222.0.75 () {56 vars in 1112 bytes} [Thu Jul 14 10:55:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:01.864324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e61d1-e6b6-485e-80c0-ff7c8323e5e2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:01.923141 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:01.968745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e61d1-e6b6-485e-80c0-ff7c8323e5e2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:02.002099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedef4b-d7a1-4e9a-8edf-2145999c4787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:02.002902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedef4b-d7a1-4e9a-8edf-2145999c4787 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:02.003282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedef4b-d7a1-4e9a-8edf-2145999c4787 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:02.004204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedef4b-d7a1-4e9a-8edf-2145999c4787 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:02.005029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedef4b-d7a1-4e9a-8edf-2145999c4787 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:02.006522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedef4b-d7a1-4e9a-8edf-2145999c4787 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:02.031426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 512/1016] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Jul 14 10:55:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:55:02.233614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18095714-f1b2-4b1b-bb92-52297e436b78 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:02.233962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18095714-f1b2-4b1b-bb92-52297e436b78 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:02.234188 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18095714-f1b2-4b1b-bb92-52297e436b78 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:02.235903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 505/1017] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:02.241549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbf9a8-cc03-40c0-b18d-5981733baaea None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:02.241851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbf9a8-cc03-40c0-b18d-5981733baaea None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:02.242143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbf9a8-cc03-40c0-b18d-5981733baaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:02.260173 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6bfbf9a8-cc03-40c0-b18d-5981733baaea None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:02.261410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6bfbf9a8-cc03-40c0-b18d-5981733baaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:02.390522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 513/1018] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:55:02.414338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18095714-f1b2-4b1b-bb92-52297e436b78 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:02.493327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:02.549915 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18095714-f1b2-4b1b-bb92-52297e436b78 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:02.588301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c9b61-5e19-498e-98e3-6ea49dfec617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:02.589101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c9b61-5e19-498e-98e3-6ea49dfec617 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:02.589383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c9b61-5e19-498e-98e3-6ea49dfec617 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:02.589658 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c9b61-5e19-498e-98e3-6ea49dfec617 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:02.590820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c9b61-5e19-498e-98e3-6ea49dfec617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:02.592330 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c9b61-5e19-498e-98e3-6ea49dfec617 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:02.594201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 506/1019] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:02.606437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbf9a8-cc03-40c0-b18d-5981733baaea None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:02.651295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:02.687707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d40d4c-4159-46af-b1c8-863f7cd84c75 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:02.688031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d40d4c-4159-46af-b1c8-863f7cd84c75 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:02.688272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d40d4c-4159-46af-b1c8-863f7cd84c75 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:02.689770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 507/1020] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:55:02.695468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385652-8134-4f81-bd11-46c0b029fa12 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:02.695762 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385652-8134-4f81-bd11-46c0b029fa12 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:02.696042 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385652-8134-4f81-bd11-46c0b029fa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:02.714030 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4e385652-8134-4f81-bd11-46c0b029fa12 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:02.715232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4e385652-8134-4f81-bd11-46c0b029fa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:02.719445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbf9a8-cc03-40c0-b18d-5981733baaea None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:02.754818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782652f-397d-4669-8c9c-a75e17e73193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:02.755633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782652f-397d-4669-8c9c-a75e17e73193 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:02.756045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782652f-397d-4669-8c9c-a75e17e73193 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:02.756323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782652f-397d-4669-8c9c-a75e17e73193 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:02.757044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782652f-397d-4669-8c9c-a75e17e73193 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:02.758472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782652f-397d-4669-8c9c-a75e17e73193 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:02.795424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 514/1021] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:55:02 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:55:02.795629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 508/1022] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:55:02.814845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782652f-397d-4669-8c9c-a75e17e73193 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:02.922794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:02.936171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782652f-397d-4669-8c9c-a75e17e73193 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:02.975903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6047b-d15f-424b-883a-05df9b02ad39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:02.976730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6047b-d15f-424b-883a-05df9b02ad39 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:02.977018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6047b-d15f-424b-883a-05df9b02ad39 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:02.977297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6047b-d15f-424b-883a-05df9b02ad39 None admin] PATH_INFO: `/v3/services/image` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:02.977970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6047b-d15f-424b-883a-05df9b02ad39 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:02.979374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6047b-d15f-424b-883a-05df9b02ad39 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-5eb6047b-d15f-424b-883a-05df9b02ad39 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:02.991646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:02.992664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:02.993516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 515/1023] 10.222.0.75 () {56 vars in 1105 bytes} [Thu Jul 14 10:55:02 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:02.999051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e385652-8134-4f81-bd11-46c0b029fa12 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:03.061602 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:03.107664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e385652-8134-4f81-bd11-46c0b029fa12 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:03.142986 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3dee-2a26-4732-987c-815d17c94a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:03.143687 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3dee-2a26-4732-987c-815d17c94a7c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:03.143983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3dee-2a26-4732-987c-815d17c94a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:03.144290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3dee-2a26-4732-987c-815d17c94a7c None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:03.145000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3dee-2a26-4732-987c-815d17c94a7c None admin] RBAC: Adding query filter params (name=image) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:03.145311 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3dee-2a26-4732-987c-815d17c94a7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:03.146699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3dee-2a26-4732-987c-815d17c94a7c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:03.156355 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 509/1024] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:03.157322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6047b-d15f-424b-883a-05df9b02ad39 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:03.269498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6047b-d15f-424b-883a-05df9b02ad39 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:03.303879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df172d-2977-48b4-a4b5-b35992056eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:03.304563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df172d-2977-48b4-a4b5-b35992056eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:03.304852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df172d-2977-48b4-a4b5-b35992056eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:03.305134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df172d-2977-48b4-a4b5-b35992056eb2 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:03.305857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df172d-2977-48b4-a4b5-b35992056eb2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:03.306179 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df172d-2977-48b4-a4b5-b35992056eb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:03.307601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df172d-2977-48b4-a4b5-b35992056eb2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:03.319362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 516/1025] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:03.320744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3dee-2a26-4732-987c-815d17c94a7c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:03.422955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3dee-2a26-4732-987c-815d17c94a7c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:03.447962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:03.460482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8466ed-6c72-4288-a547-58475bef0c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:03.461149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8466ed-6c72-4288-a547-58475bef0c01 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:03.461437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8466ed-6c72-4288-a547-58475bef0c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:03.461721 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8466ed-6c72-4288-a547-58475bef0c01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:03.462511 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8466ed-6c72-4288-a547-58475bef0c01 None admin] RBAC: Adding query filter params (interface=public, service_id=1519d000300b4061bc3bb6f7c7e53e57, region_id=RegionOne) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:03.462829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8466ed-6c72-4288-a547-58475bef0c01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:03.464234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8466ed-6c72-4288-a547-58475bef0c01 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:03.474256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 510/1026] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Jul 14 10:55:03 2022] GET /identity/v3/endpoints?service_id=1519d000300b4061bc3bb6f7c7e53e57&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:03.475495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df172d-2977-48b4-a4b5-b35992056eb2 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:03.496322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:03.579730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df172d-2977-48b4-a4b5-b35992056eb2 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:03.614122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b66475-d9d2-4239-839d-1db78df36204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:03.614789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b66475-d9d2-4239-839d-1db78df36204 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:03.615076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b66475-d9d2-4239-839d-1db78df36204 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:03.615356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b66475-d9d2-4239-839d-1db78df36204 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:03.616066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b66475-d9d2-4239-839d-1db78df36204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:03.617439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b66475-d9d2-4239-839d-1db78df36204 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:03.631658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 517/1027] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Jul 14 10:55:03 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:55:04.250594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182bbd2-382d-452a-976f-a19d12b2e49c None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:04.251091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182bbd2-382d-452a-976f-a19d12b2e49c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:04.251424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182bbd2-382d-452a-976f-a19d12b2e49c None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:04.252984 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 511/1028] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:04.258731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc4ad6-382a-433f-9a7f-31b6c5885c0d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:04.259024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc4ad6-382a-433f-9a7f-31b6c5885c0d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:04.259312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc4ad6-382a-433f-9a7f-31b6c5885c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:04.277461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b5cc4ad6-382a-433f-9a7f-31b6c5885c0d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:04.278685 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b5cc4ad6-382a-433f-9a7f-31b6c5885c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:04.349470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 518/1029] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:55:04.372984 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182bbd2-382d-452a-976f-a19d12b2e49c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:04.477896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182bbd2-382d-452a-976f-a19d12b2e49c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:04.512087 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058cd75-1fc3-48f3-8af1-b5bf09655e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:04.512904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058cd75-1fc3-48f3-8af1-b5bf09655e62 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:04.513191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058cd75-1fc3-48f3-8af1-b5bf09655e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:04.513470 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058cd75-1fc3-48f3-8af1-b5bf09655e62 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:04.524618 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058cd75-1fc3-48f3-8af1-b5bf09655e62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-a058cd75-1fc3-48f3-8af1-b5bf09655e62 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:55:04.532747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:04.533676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:04.534523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:04.534523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:04.534523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:04.534523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:04.534523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:55:04.534523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:55:04.534523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:55:04.534523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:55:04.534523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:55:04.534523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:55:04.534523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:55:04.534523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:04.535818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 512/1030] 10.222.0.75 () {56 vars in 1120 bytes} [Thu Jul 14 10:55:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:04.540012 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc4ad6-382a-433f-9a7f-31b6c5885c0d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:04.554509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:04.646283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc4ad6-382a-433f-9a7f-31b6c5885c0d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:04.681130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c3216-d0e1-4177-bfc1-da46d0f33dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:04.681815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c3216-d0e1-4177-bfc1-da46d0f33dac None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:04.682105 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c3216-d0e1-4177-bfc1-da46d0f33dac None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:04.682391 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c3216-d0e1-4177-bfc1-da46d0f33dac None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:04.683763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c3216-d0e1-4177-bfc1-da46d0f33dac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:04.683763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c3216-d0e1-4177-bfc1-da46d0f33dac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:04.685191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c3216-d0e1-4177-bfc1-da46d0f33dac None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:04.699082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 519/1031] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Jul 14 10:55:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:04.700294 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058cd75-1fc3-48f3-8af1-b5bf09655e62 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:04.749381 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:04.806045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058cd75-1fc3-48f3-8af1-b5bf09655e62 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:04.840283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db5ff5-49c8-4dbe-8f30-8c1e20ea6095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:04.841076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db5ff5-49c8-4dbe-8f30-8c1e20ea6095 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:04.841358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db5ff5-49c8-4dbe-8f30-8c1e20ea6095 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:04.841633 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db5ff5-49c8-4dbe-8f30-8c1e20ea6095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:04.852809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db5ff5-49c8-4dbe-8f30-8c1e20ea6095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:04.854397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db5ff5-49c8-4dbe-8f30-8c1e20ea6095 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-e6db5ff5-49c8-4dbe-8f30-8c1e20ea6095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:55:04.872657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:04.873616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:04.879916 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:55:04.879916 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:55:04.879916 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:04.879916 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:04.879916 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 513/1032] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:04.881630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c3216-d0e1-4177-bfc1-da46d0f33dac None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:04.909034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d713736-74cd-4175-bc4b-02ecb0acc2b7 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:04.909333 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d713736-74cd-4175-bc4b-02ecb0acc2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:04.909647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d713736-74cd-4175-bc4b-02ecb0acc2b7 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:04.911929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 514/1033] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:55:04.916931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515d9d7-c1fb-43df-8f8f-44c0ba0f4164 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:04.917216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515d9d7-c1fb-43df-8f8f-44c0ba0f4164 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:04.917499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515d9d7-c1fb-43df-8f8f-44c0ba0f4164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:04.935622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5515d9d7-c1fb-43df-8f8f-44c0ba0f4164 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:04.936831 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5515d9d7-c1fb-43df-8f8f-44c0ba0f4164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:04.968131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:04.997016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c3216-d0e1-4177-bfc1-da46d0f33dac None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:05.010846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 515/1034] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:55:05.031814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e522e-f001-4888-8929-a700d117a35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:05.032489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e522e-f001-4888-8929-a700d117a35c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:05.032777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e522e-f001-4888-8929-a700d117a35c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:05.033066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e522e-f001-4888-8929-a700d117a35c None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:05.033773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e522e-f001-4888-8929-a700d117a35c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:05.034085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e522e-f001-4888-8929-a700d117a35c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:05.036040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e522e-f001-4888-8929-a700d117a35c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:05.036164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515d9d7-c1fb-43df-8f8f-44c0ba0f4164 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:05.053609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 520/1035] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:05.054674 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e522e-f001-4888-8929-a700d117a35c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:05.070203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:05.154853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515d9d7-c1fb-43df-8f8f-44c0ba0f4164 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:05.168438 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e522e-f001-4888-8929-a700d117a35c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:05.192079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595db60-e3de-47be-a058-14f8e40291e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:05.192888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595db60-e3de-47be-a058-14f8e40291e7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:05.193172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595db60-e3de-47be-a058-14f8e40291e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:05.193450 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595db60-e3de-47be-a058-14f8e40291e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:05.195587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595db60-e3de-47be-a058-14f8e40291e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:05.197048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595db60-e3de-47be-a058-14f8e40291e7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:05.198712 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 516/1036] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:05.204254 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595db60-e3de-47be-a058-14f8e40291e7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:05.215473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e29b3-2e59-4aa8-80c8-746560503e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:05.216617 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e29b3-2e59-4aa8-80c8-746560503e70 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:05.216994 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e29b3-2e59-4aa8-80c8-746560503e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:05.217268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e29b3-2e59-4aa8-80c8-746560503e70 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:05.219351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:05.226866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e29b3-2e59-4aa8-80c8-746560503e70 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:05.228936 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e29b3-2e59-4aa8-80c8-746560503e70 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-d53e29b3-2e59-4aa8-80c8-746560503e70 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:55:05.244728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:05.245713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:05.246823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:05.246823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:05.246823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:55:05.246823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:55:05.246823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:55:05.246823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:55:05.246823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:55:05.246823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:55:05.246823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:55:05.246823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:05.246823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 521/1037] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Jul 14 10:55:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:05.251504 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e29b3-2e59-4aa8-80c8-746560503e70 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:05.309682 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:05.316122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595db60-e3de-47be-a058-14f8e40291e7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:05.352394 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b1bff-a625-4e2a-ae97-64ad59024033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:05.353135 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b1bff-a625-4e2a-ae97-64ad59024033 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:05.353419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b1bff-a625-4e2a-ae97-64ad59024033 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:05.353695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b1bff-a625-4e2a-ae97-64ad59024033 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:05.354406 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b1bff-a625-4e2a-ae97-64ad59024033 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:05.355889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b1bff-a625-4e2a-ae97-64ad59024033 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:05.360899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e29b3-2e59-4aa8-80c8-746560503e70 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:05.387764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 517/1038] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:55:05 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:55:05.400884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01807f39-2e51-492a-8a02-874bd57c1676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:05.401585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01807f39-2e51-492a-8a02-874bd57c1676 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:05.401866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01807f39-2e51-492a-8a02-874bd57c1676 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:05.402142 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01807f39-2e51-492a-8a02-874bd57c1676 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:05.403016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01807f39-2e51-492a-8a02-874bd57c1676 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:05.403328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01807f39-2e51-492a-8a02-874bd57c1676 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:05.404869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01807f39-2e51-492a-8a02-874bd57c1676 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:05.425630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 522/1039] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Jul 14 10:55:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:05.431749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b1bff-a625-4e2a-ae97-64ad59024033 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:05.554080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b1bff-a625-4e2a-ae97-64ad59024033 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:05.591708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831f961-2c89-42d7-bbf3-659f1806bb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:05.592292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831f961-2c89-42d7-bbf3-659f1806bb7d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:05.592574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831f961-2c89-42d7-bbf3-659f1806bb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:05.592850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831f961-2c89-42d7-bbf3-659f1806bb7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:05.615639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:05.664189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831f961-2c89-42d7-bbf3-659f1806bb7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:05.665664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831f961-2c89-42d7-bbf3-659f1806bb7d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:05.692596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b3fe8-d3dc-48f3-a3bb-23de5d8e7538 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:05.693143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b3fe8-d3dc-48f3-a3bb-23de5d8e7538 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:05.693143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b3fe8-d3dc-48f3-a3bb-23de5d8e7538 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:05.694487 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 523/1040] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:55:05.699728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faff6c-ec83-4d3e-b913-b07b2d52756c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:05.700019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faff6c-ec83-4d3e-b913-b07b2d52756c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:05.700299 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faff6c-ec83-4d3e-b913-b07b2d52756c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:05.716989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-72faff6c-ec83-4d3e-b913-b07b2d52756c None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:05.718175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-72faff6c-ec83-4d3e-b913-b07b2d52756c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:05.770213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:05.776502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 518/1041] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:55:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:55:05.782706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831f961-2c89-42d7-bbf3-659f1806bb7d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:05.795613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 524/1042] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:55:05.818861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72faff6c-ec83-4d3e-b913-b07b2d52756c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:05.840760 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:05.897588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831f961-2c89-42d7-bbf3-659f1806bb7d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:05.932875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72faff6c-ec83-4d3e-b913-b07b2d52756c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:05.939803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc866a-0dbb-4ac7-a370-ce9c3f909a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:05.940627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc866a-0dbb-4ac7-a370-ce9c3f909a7a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:05.941457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc866a-0dbb-4ac7-a370-ce9c3f909a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:05.941743 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc866a-0dbb-4ac7-a370-ce9c3f909a7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:05.953354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc866a-0dbb-4ac7-a370-ce9c3f909a7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:05.954867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc866a-0dbb-4ac7-a370-ce9c3f909a7a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-a7bc866a-0dbb-4ac7-a370-ce9c3f909a7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:55:05.974387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:05.980577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:05.981580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:55:05.981580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:55:05.981580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:05.981580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:05.981580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 519/1043] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:55:05.981580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc866a-0dbb-4ac7-a370-ce9c3f909a7a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:05.984454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e502e6-0137-49c6-b8f7-3dda4eceb05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:05.985284 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e502e6-0137-49c6-b8f7-3dda4eceb05d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:05.985571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e502e6-0137-49c6-b8f7-3dda4eceb05d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:05.985855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e502e6-0137-49c6-b8f7-3dda4eceb05d None admin] PATH_INFO: `/v3/services/image` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:05.986520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e502e6-0137-49c6-b8f7-3dda4eceb05d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:05.988016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e502e6-0137-49c6-b8f7-3dda4eceb05d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-29e502e6-0137-49c6-b8f7-3dda4eceb05d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:06.000542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:06.001632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:06.004259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 525/1044] 10.222.0.75 () {56 vars in 1105 bytes} [Thu Jul 14 10:55:05 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:55:06.008648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e502e6-0137-49c6-b8f7-3dda4eceb05d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:06.059926 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:06.109812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc866a-0dbb-4ac7-a370-ce9c3f909a7a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:06.113257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:06.120410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e502e6-0137-49c6-b8f7-3dda4eceb05d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:06.149454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8a6ef-30b7-4d6f-a05d-88d7f5720033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:06.150064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8a6ef-30b7-4d6f-a05d-88d7f5720033 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:06.150329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8a6ef-30b7-4d6f-a05d-88d7f5720033 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:06.150611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8a6ef-30b7-4d6f-a05d-88d7f5720033 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:06.151336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8a6ef-30b7-4d6f-a05d-88d7f5720033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:06.151741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8a6ef-30b7-4d6f-a05d-88d7f5720033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:06.153146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8a6ef-30b7-4d6f-a05d-88d7f5720033 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:06.156949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edae8ef-6868-4891-9431-a26ae5d51b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:06.157634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edae8ef-6868-4891-9431-a26ae5d51b22 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:06.157923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edae8ef-6868-4891-9431-a26ae5d51b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:06.158207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edae8ef-6868-4891-9431-a26ae5d51b22 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:06.158922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edae8ef-6868-4891-9431-a26ae5d51b22 None admin] RBAC: Adding query filter params (name=image) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:06.159236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edae8ef-6868-4891-9431-a26ae5d51b22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:06.161291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edae8ef-6868-4891-9431-a26ae5d51b22 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:06.166358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 520/1045] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:06.167503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 526/1046] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:06.171837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8a6ef-30b7-4d6f-a05d-88d7f5720033 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:06.172830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edae8ef-6868-4891-9431-a26ae5d51b22 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:06.286934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edae8ef-6868-4891-9431-a26ae5d51b22 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:06.290625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8a6ef-30b7-4d6f-a05d-88d7f5720033 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:06.298153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:06.325001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675638fb-cc93-4cf9-8ae7-1b28f6e8c32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:06.325677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675638fb-cc93-4cf9-8ae7-1b28f6e8c32e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:06.325975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675638fb-cc93-4cf9-8ae7-1b28f6e8c32e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:06.326248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675638fb-cc93-4cf9-8ae7-1b28f6e8c32e None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:06.326787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075948b-196d-4477-9c0e-6e210844baf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:06.326982 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675638fb-cc93-4cf9-8ae7-1b28f6e8c32e None admin] RBAC: Adding query filter params (type=image) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:06.327259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675638fb-cc93-4cf9-8ae7-1b28f6e8c32e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:06.327626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075948b-196d-4477-9c0e-6e210844baf9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:06.327920 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075948b-196d-4477-9c0e-6e210844baf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:06.328207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075948b-196d-4477-9c0e-6e210844baf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:06.329303 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675638fb-cc93-4cf9-8ae7-1b28f6e8c32e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:06.336280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075948b-196d-4477-9c0e-6e210844baf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:06.337763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075948b-196d-4477-9c0e-6e210844baf9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:06.342854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 527/1047] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:06.343658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675638fb-cc93-4cf9-8ae7-1b28f6e8c32e None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-4075948b-196d-4477-9c0e-6e210844baf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:55:06.354215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:06.355172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:06.356993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:55:06.356993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:55:06.356993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:55:06.356993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:55:06.356993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:55:06.356993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:55:06.356993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:55:06.356993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:06.356993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 521/1048] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:55:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:55:06.361167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075948b-196d-4477-9c0e-6e210844baf9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:06.388457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:06.455614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675638fb-cc93-4cf9-8ae7-1b28f6e8c32e None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:06.475673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075948b-196d-4477-9c0e-6e210844baf9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:06.493150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3ff46-3d25-490e-aafb-a97a05a02c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:06.493936 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3ff46-3d25-490e-aafb-a97a05a02c4a None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:06.494221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3ff46-3d25-490e-aafb-a97a05a02c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:06.494613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3ff46-3d25-490e-aafb-a97a05a02c4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:06.495357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3ff46-3d25-490e-aafb-a97a05a02c4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:06.496765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3ff46-3d25-490e-aafb-a97a05a02c4a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:06.514416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a503fb8-9083-4470-b3ae-cf118da6f2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:06.515090 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a503fb8-9083-4470-b3ae-cf118da6f2bd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:06.515408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a503fb8-9083-4470-b3ae-cf118da6f2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:06.515692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a503fb8-9083-4470-b3ae-cf118da6f2bd None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:06.516481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a503fb8-9083-4470-b3ae-cf118da6f2bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:06.516798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a503fb8-9083-4470-b3ae-cf118da6f2bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:06.527534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 528/1049] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Jul 14 10:55:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:55:06.527737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a503fb8-9083-4470-b3ae-cf118da6f2bd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:06.532055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 522/1050] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:55:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:06.538401 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3ff46-3d25-490e-aafb-a97a05a02c4a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:06.576062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:06.665363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3ff46-3d25-490e-aafb-a97a05a02c4a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:06.703860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3e62b-ac8c-4967-a7cd-33e5fe464cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:06.704615 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3e62b-ac8c-4967-a7cd-33e5fe464cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:06.704804 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3e62b-ac8c-4967-a7cd-33e5fe464cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:06.705081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3e62b-ac8c-4967-a7cd-33e5fe464cb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:06.705970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3e62b-ac8c-4967-a7cd-33e5fe464cb9 None admin] RBAC: Adding query filter params (role.id=4d8a62dacbc5492b9fa4b11c57673134, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=7e572bf785284a8693cf386ced004524) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:06.706235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3e62b-ac8c-4967-a7cd-33e5fe464cb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:06.708275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3e62b-ac8c-4967-a7cd-33e5fe464cb9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:06.714465 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 529/1051] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:55:06 2022] GET /identity/v3/role_assignments?user.id=7e572bf785284a8693cf386ced004524&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=4d8a62dacbc5492b9fa4b11c57673134 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:07.533047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060213e3-1c3d-46b3-9aa7-12c41dea2e93 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:07.533543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060213e3-1c3d-46b3-9aa7-12c41dea2e93 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:07.533543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060213e3-1c3d-46b3-9aa7-12c41dea2e93 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:07.535470 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 523/1052] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:07.540557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5b70e-27e0-4a12-ace0-f304b595a9d4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:07.540859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5b70e-27e0-4a12-ace0-f304b595a9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:07.541145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5b70e-27e0-4a12-ace0-f304b595a9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:07.558693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-02f5b70e-27e0-4a12-ace0-f304b595a9d4 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:07.559930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-02f5b70e-27e0-4a12-ace0-f304b595a9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:07.628218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 530/1053] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:07.651582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060213e3-1c3d-46b3-9aa7-12c41dea2e93 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:07.740262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:07.757848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060213e3-1c3d-46b3-9aa7-12c41dea2e93 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:07.792395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bca70-9f12-4e9e-bf22-1d37d3970cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:07.793171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bca70-9f12-4e9e-bf22-1d37d3970cfe None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:07.793456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bca70-9f12-4e9e-bf22-1d37d3970cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:07.793734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bca70-9f12-4e9e-bf22-1d37d3970cfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:07.805313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bca70-9f12-4e9e-bf22-1d37d3970cfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-815bca70-9f12-4e9e-bf22-1d37d3970cfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:55:07.813354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:07.814344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:07.815216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:07.815216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:07.815216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:07.815216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:07.815216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:55:07.815216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:55:07.815216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:55:07.815216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:55:07.815216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:55:07.815216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:55:07.815216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:07.815216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:07.822749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 524/1054] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:55:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:07.822889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5b70e-27e0-4a12-ace0-f304b595a9d4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:07.843220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:07.922361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5b70e-27e0-4a12-ace0-f304b595a9d4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:07.957264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803d5c-6ec1-4ada-8349-75293d4abe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:07.957944 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803d5c-6ec1-4ada-8349-75293d4abe05 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:07.958233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803d5c-6ec1-4ada-8349-75293d4abe05 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:07.958511 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803d5c-6ec1-4ada-8349-75293d4abe05 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:07.959255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e803d5c-6ec1-4ada-8349-75293d4abe05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:07.959593 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e803d5c-6ec1-4ada-8349-75293d4abe05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:07.961105 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e803d5c-6ec1-4ada-8349-75293d4abe05 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:07.969933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 531/1055] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:55:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:07.976195 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bca70-9f12-4e9e-bf22-1d37d3970cfe None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:08.079991 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bca70-9f12-4e9e-bf22-1d37d3970cfe None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:08.113214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc8ebf-641c-4805-a87b-dee1c472856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:08.114088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc8ebf-641c-4805-a87b-dee1c472856a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:08.114373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc8ebf-641c-4805-a87b-dee1c472856a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:08.114649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc8ebf-641c-4805-a87b-dee1c472856a None admin] PATH_INFO: `/v3/users/4e3a2f1d31524717bf9cf49331b28e47` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:08.127881 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc8ebf-641c-4805-a87b-dee1c472856a None admin] RBAC: Authorizing `identity:get_user(user_id=4e3a2f1d31524717bf9cf49331b28e47)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:08.129871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc8ebf-641c-4805-a87b-dee1c472856a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:08.131716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 525/1056] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:07 2022] GET /identity/v3/users/4e3a2f1d31524717bf9cf49331b28e47 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:08.137231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803d5c-6ec1-4ada-8349-75293d4abe05 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:08.227577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:08.246684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803d5c-6ec1-4ada-8349-75293d4abe05 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:08.281351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6834d2e-a7ee-499d-8f4f-2f54ad815d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:08.282006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6834d2e-a7ee-499d-8f4f-2f54ad815d24 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:08.282292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6834d2e-a7ee-499d-8f4f-2f54ad815d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:08.282571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6834d2e-a7ee-499d-8f4f-2f54ad815d24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:08.351099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6834d2e-a7ee-499d-8f4f-2f54ad815d24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:08.352498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6834d2e-a7ee-499d-8f4f-2f54ad815d24 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:08.435894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:08.461515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 532/1057] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:55:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:55:08.462832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc8ebf-641c-4805-a87b-dee1c472856a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:08.465994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:08.567008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc8ebf-641c-4805-a87b-dee1c472856a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:08.591155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09428137-9cf2-414d-8922-7b8a7131d98d None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:08.591479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09428137-9cf2-414d-8922-7b8a7131d98d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:08.591762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09428137-9cf2-414d-8922-7b8a7131d98d None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:08.593137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 533/1058] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:55:08.598262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6445625-8aa0-40fd-9780-8f71a3b62fc2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:08.598623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6445625-8aa0-40fd-9780-8f71a3b62fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:08.598839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6445625-8aa0-40fd-9780-8f71a3b62fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:08.601590 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6789b4-e1ce-4fe3-888b-366dce79d4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:08.602402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6789b4-e1ce-4fe3-888b-366dce79d4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:08.602684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6789b4-e1ce-4fe3-888b-366dce79d4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:08.602958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6789b4-e1ce-4fe3-888b-366dce79d4c0 None admin] PATH_INFO: `/v3/projects/7bcccf0364454757a67b6b769795f701` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:08.604168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6789b4-e1ce-4fe3-888b-366dce79d4c0 None admin] RBAC: Authorizing `identity:get_project(project_id=7bcccf0364454757a67b6b769795f701)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:08.605618 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6789b4-e1ce-4fe3-888b-366dce79d4c0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:08.612365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 526/1059] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:55:08 2022] GET /identity/v3/projects/7bcccf0364454757a67b6b769795f701 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:08.613817 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6789b4-e1ce-4fe3-888b-366dce79d4c0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:08.619137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b6445625-8aa0-40fd-9780-8f71a3b62fc2 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:08.620454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b6445625-8aa0-40fd-9780-8f71a3b62fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:08.689696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:08.696361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 534/1060] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:08.718360 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6445625-8aa0-40fd-9780-8f71a3b62fc2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:08.727534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6789b4-e1ce-4fe3-888b-366dce79d4c0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:08.768100 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477c783-91bf-4b7e-bd28-e6297db7246d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:08.768804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477c783-91bf-4b7e-bd28-e6297db7246d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:08.769458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477c783-91bf-4b7e-bd28-e6297db7246d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:08.769754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477c783-91bf-4b7e-bd28-e6297db7246d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:08.770560 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477c783-91bf-4b7e-bd28-e6297db7246d None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=7bcccf0364454757a67b6b769795f701, user.id=4e3a2f1d31524717bf9cf49331b28e47) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:08.770878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477c783-91bf-4b7e-bd28-e6297db7246d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:08.772341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477c783-91bf-4b7e-bd28-e6297db7246d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:08.777976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 527/1061] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:55:08 2022] GET /identity/v3/role_assignments?user.id=4e3a2f1d31524717bf9cf49331b28e47&scope.project.id=7bcccf0364454757a67b6b769795f701&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:08.779249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64245ba-25b3-41da-a1ab-62753282e717 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:08.779566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64245ba-25b3-41da-a1ab-62753282e717 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:08.779853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64245ba-25b3-41da-a1ab-62753282e717 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:08.781432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 528/1062] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:08.787724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2fc8-5886-40ef-935f-738a80d9f56b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:08.788045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2fc8-5886-40ef-935f-738a80d9f56b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:08.788327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2fc8-5886-40ef-935f-738a80d9f56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:08.806739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b10a2fc8-5886-40ef-935f-738a80d9f56b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:08.808280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b10a2fc8-5886-40ef-935f-738a80d9f56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:08.829752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6445625-8aa0-40fd-9780-8f71a3b62fc2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:08.844959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:08.873080 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6cf91-1dc3-41bc-aad4-93a4d757d2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:08.873734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6cf91-1dc3-41bc-aad4-93a4d757d2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:08.874023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6cf91-1dc3-41bc-aad4-93a4d757d2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:08.874351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6cf91-1dc3-41bc-aad4-93a4d757d2f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:08.887879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 529/1063] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:55:08.911497 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a2fc8-5886-40ef-935f-738a80d9f56b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:08.951151 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6cf91-1dc3-41bc-aad4-93a4d757d2f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:08.953957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6cf91-1dc3-41bc-aad4-93a4d757d2f4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:08.981717 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:09.023756 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a2fc8-5886-40ef-935f-738a80d9f56b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:09.065598 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 535/1064] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:55:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:55:09.065835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118039a-c9f0-4d5b-9550-6cd52a7b811e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:09.065835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118039a-c9f0-4d5b-9550-6cd52a7b811e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:09.065835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118039a-c9f0-4d5b-9550-6cd52a7b811e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:09.065835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118039a-c9f0-4d5b-9550-6cd52a7b811e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:09.067641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6cf91-1dc3-41bc-aad4-93a4d757d2f4 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:09.079940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118039a-c9f0-4d5b-9550-6cd52a7b811e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:55:09.082684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-4118039a-c9f0-4d5b-9550-6cd52a7b811e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:55:09.091054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:09.092088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:09.092924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 530/1065] 10.222.0.75 () {56 vars in 1120 bytes} [Thu Jul 14 10:55:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:55:09.100371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118039a-c9f0-4d5b-9550-6cd52a7b811e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:09.195100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6cf91-1dc3-41bc-aad4-93a4d757d2f4 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:09.219593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118039a-c9f0-4d5b-9550-6cd52a7b811e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:09.234368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0245109-4bca-47d2-8ef9-3551976c4115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:09.235303 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0245109-4bca-47d2-8ef9-3551976c4115 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:09.235616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0245109-4bca-47d2-8ef9-3551976c4115 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:09.235899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0245109-4bca-47d2-8ef9-3551976c4115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:09.246823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:09.248014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0245109-4bca-47d2-8ef9-3551976c4115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:09.249484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0245109-4bca-47d2-8ef9-3551976c4115 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:09.258527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d709c-55c0-4e7f-ab41-f5ec08018809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:09.259232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d709c-55c0-4e7f-ab41-f5ec08018809 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:09.259877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d709c-55c0-4e7f-ab41-f5ec08018809 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:09.260392 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d709c-55c0-4e7f-ab41-f5ec08018809 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:09.261186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d709c-55c0-4e7f-ab41-f5ec08018809 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:09.261805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d709c-55c0-4e7f-ab41-f5ec08018809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:09.263466 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d709c-55c0-4e7f-ab41-f5ec08018809 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:09.274283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 531/1066] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Jul 14 10:55:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-d0245109-4bca-47d2-8ef9-3551976c4115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:55:09.275764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:09.276616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:09.277496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:55:09.277496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:55:09.277496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:09.277496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:09.277496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 536/1067] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:55:09.279789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d709c-55c0-4e7f-ab41-f5ec08018809 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:09.282945 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0245109-4bca-47d2-8ef9-3551976c4115 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:09.353511 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:09.384551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d709c-55c0-4e7f-ab41-f5ec08018809 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:09.397416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0245109-4bca-47d2-8ef9-3551976c4115 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:09.420723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d44f28-ae8e-4b46-a998-055560bf0f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:09.421526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d44f28-ae8e-4b46-a998-055560bf0f28 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:09.421807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d44f28-ae8e-4b46-a998-055560bf0f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:09.422075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d44f28-ae8e-4b46-a998-055560bf0f28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:09.433662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d44f28-ae8e-4b46-a998-055560bf0f28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:09.435113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d44f28-ae8e-4b46-a998-055560bf0f28 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:09.438596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff8ac4-1f18-4a42-8630-a72719e1af2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:09.439482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff8ac4-1f18-4a42-8630-a72719e1af2f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:09.439922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff8ac4-1f18-4a42-8630-a72719e1af2f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:09.440349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff8ac4-1f18-4a42-8630-a72719e1af2f None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:09.441794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff8ac4-1f18-4a42-8630-a72719e1af2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:09.442678 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff8ac4-1f18-4a42-8630-a72719e1af2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:09.444634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff8ac4-1f18-4a42-8630-a72719e1af2f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-91d44f28-ae8e-4b46-a998-055560bf0f28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:55:09.452848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:09.453930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:09.457182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:55:09.457182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:55:09.457182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:09.457182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:09.457182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 532/1068] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:09.461098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d44f28-ae8e-4b46-a998-055560bf0f28 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:09.463837 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 537/1069] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:09.508791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:09.577425 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d44f28-ae8e-4b46-a998-055560bf0f28 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:09.615490 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbeb21f-589a-4e9a-8787-52f020a4fe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:09.616194 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbeb21f-589a-4e9a-8787-52f020a4fe27 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:09.616438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbeb21f-589a-4e9a-8787-52f020a4fe27 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:09.616709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbeb21f-589a-4e9a-8787-52f020a4fe27 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:09.617434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbeb21f-589a-4e9a-8787-52f020a4fe27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:09.617740 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbeb21f-589a-4e9a-8787-52f020a4fe27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:09.619351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbeb21f-589a-4e9a-8787-52f020a4fe27 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:09.634428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 533/1070] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:09.640192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff8ac4-1f18-4a42-8630-a72719e1af2f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:09.756546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff8ac4-1f18-4a42-8630-a72719e1af2f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:09.782372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:09.793451 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819fd87-11fd-4059-b650-ac040a2ec9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:09.794254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819fd87-11fd-4059-b650-ac040a2ec9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:09.794537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819fd87-11fd-4059-b650-ac040a2ec9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:09.794814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819fd87-11fd-4059-b650-ac040a2ec9b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:09.802640 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819fd87-11fd-4059-b650-ac040a2ec9b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:09.804138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819fd87-11fd-4059-b650-ac040a2ec9b3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-d819fd87-11fd-4059-b650-ac040a2ec9b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:55:09.818839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:09.819869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:09.821153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:09.821153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:09.821153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:55:09.821153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:55:09.821153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:55:09.821153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:55:09.821153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:55:09.821153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:55:09.821153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:55:09.821153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:09.821153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 538/1071] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Jul 14 10:55:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:09.825827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbeb21f-589a-4e9a-8787-52f020a4fe27 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:09.923559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:09.928288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbeb21f-589a-4e9a-8787-52f020a4fe27 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:09.963179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48028c-1251-4031-95c3-ed6ccb4c0c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:09.963886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48028c-1251-4031-95c3-ed6ccb4c0c99 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:09.964170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48028c-1251-4031-95c3-ed6ccb4c0c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:09.964446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48028c-1251-4031-95c3-ed6ccb4c0c99 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:09.965279 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48028c-1251-4031-95c3-ed6ccb4c0c99 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:09.965585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48028c-1251-4031-95c3-ed6ccb4c0c99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:09.967018 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48028c-1251-4031-95c3-ed6ccb4c0c99 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:09.984505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 534/1072] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Jul 14 10:55:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:09.990851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819fd87-11fd-4059-b650-ac040a2ec9b3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:10.089792 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819fd87-11fd-4059-b650-ac040a2ec9b3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:10.131264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861aaf0-c2e9-4b19-9992-374bf2f2b873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:10.132091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861aaf0-c2e9-4b19-9992-374bf2f2b873 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:10.132373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861aaf0-c2e9-4b19-9992-374bf2f2b873 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:10.132649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861aaf0-c2e9-4b19-9992-374bf2f2b873 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:10.143374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861aaf0-c2e9-4b19-9992-374bf2f2b873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:10.144840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861aaf0-c2e9-4b19-9992-374bf2f2b873 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-0861aaf0-c2e9-4b19-9992-374bf2f2b873 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:55:10.161898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:10.162989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:10.164274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:55:10.164274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:55:10.164274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:10.164274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:10.164274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 539/1073] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:10.168752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48028c-1251-4031-95c3-ed6ccb4c0c99 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:10.270814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48028c-1251-4031-95c3-ed6ccb4c0c99 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:10.305357 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11faa4-cbdd-4568-9576-ba920f881fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:10.306017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11faa4-cbdd-4568-9576-ba920f881fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:10.306298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11faa4-cbdd-4568-9576-ba920f881fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:10.306578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11faa4-cbdd-4568-9576-ba920f881fa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:10.307286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11faa4-cbdd-4568-9576-ba920f881fa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:10.307617 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11faa4-cbdd-4568-9576-ba920f881fa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:10.309242 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11faa4-cbdd-4568-9576-ba920f881fa0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:10.320552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 535/1074] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:10.325990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861aaf0-c2e9-4b19-9992-374bf2f2b873 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:10.362158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:10.428406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861aaf0-c2e9-4b19-9992-374bf2f2b873 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:10.463301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155ee70-47cd-4912-a7aa-aa708652a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:10.464133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155ee70-47cd-4912-a7aa-aa708652a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:10.464424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155ee70-47cd-4912-a7aa-aa708652a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:10.464707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155ee70-47cd-4912-a7aa-aa708652a6e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:10.471567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155ee70-47cd-4912-a7aa-aa708652a6e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:10.473534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155ee70-47cd-4912-a7aa-aa708652a6e8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a155ee70-47cd-4912-a7aa-aa708652a6e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:55:10.487217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:10.488851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:10.489740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:55:10.489740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:55:10.489740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:55:10.489740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:55:10.489740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:55:10.489740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:55:10.489740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:55:10.489740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:10.489740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 540/1075] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:10.494784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11faa4-cbdd-4568-9576-ba920f881fa0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:10.513024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:10.593619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11faa4-cbdd-4568-9576-ba920f881fa0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:10.626467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4968b12-515d-4d5b-93bf-c4466d37bcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:10.627046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4968b12-515d-4d5b-93bf-c4466d37bcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:10.627327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4968b12-515d-4d5b-93bf-c4466d37bcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:10.627631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4968b12-515d-4d5b-93bf-c4466d37bcd9 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:10.628433 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4968b12-515d-4d5b-93bf-c4466d37bcd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:10.628738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4968b12-515d-4d5b-93bf-c4466d37bcd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:10.630160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4968b12-515d-4d5b-93bf-c4466d37bcd9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:10.641737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 536/1076] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:55:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:10.648334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155ee70-47cd-4912-a7aa-aa708652a6e8 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:10.745985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:10.752727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155ee70-47cd-4912-a7aa-aa708652a6e8 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:10.786762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d45c61-a8fb-4660-8cf4-7d6bcc06365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:10.787692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d45c61-a8fb-4660-8cf4-7d6bcc06365d None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:10.787988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d45c61-a8fb-4660-8cf4-7d6bcc06365d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:10.788269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d45c61-a8fb-4660-8cf4-7d6bcc06365d None admin] PATH_INFO: `/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/7e572bf785284a8693cf386ced004524/roles/4d8a62dacbc5492b9fa4b11c57673134` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:10.790516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d45c61-a8fb-4660-8cf4-7d6bcc06365d None admin] RBAC: Authorizing `identity:create_grant(project_id=a68601f16d1c49548b6e3af0b9819ba3, user_id=7e572bf785284a8693cf386ced004524, role_id=4d8a62dacbc5492b9fa4b11c57673134)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:10.791984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d45c61-a8fb-4660-8cf4-7d6bcc06365d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:10.804736 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 541/1077] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:55:10 2022] PUT /identity/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/7e572bf785284a8693cf386ced004524/roles/4d8a62dacbc5492b9fa4b11c57673134 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:55:10.839196 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6504dd6-8b34-4ceb-a56e-dcf9206e9639 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:10.839513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6504dd6-8b34-4ceb-a56e-dcf9206e9639 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:10.839791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6504dd6-8b34-4ceb-a56e-dcf9206e9639 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:10.841194 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 537/1078] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:10.847747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb2a1e-bb91-4d48-bb66-278a5a2f3670 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:10.848050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb2a1e-bb91-4d48-bb66-278a5a2f3670 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:10.848334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb2a1e-bb91-4d48-bb66-278a5a2f3670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:10.867645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0ffb2a1e-bb91-4d48-bb66-278a5a2f3670 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:10.868902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0ffb2a1e-bb91-4d48-bb66-278a5a2f3670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:10.941709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 542/1079] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:10.967130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6504dd6-8b34-4ceb-a56e-dcf9206e9639 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:11.044267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:11.068231 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6504dd6-8b34-4ceb-a56e-dcf9206e9639 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:11.101825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e170f-7141-425c-813e-da075aaca526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:11.102625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e170f-7141-425c-813e-da075aaca526 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:11.102913 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e170f-7141-425c-813e-da075aaca526 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:11.103192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e170f-7141-425c-813e-da075aaca526 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:11.113286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e170f-7141-425c-813e-da075aaca526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-b78e170f-7141-425c-813e-da075aaca526 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:55:11.121005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:11.121911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:11.122755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 538/1080] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:55:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:11.128896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb2a1e-bb91-4d48-bb66-278a5a2f3670 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:11.223406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:11.242207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb2a1e-bb91-4d48-bb66-278a5a2f3670 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:11.276880 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c41407-3e88-4c8b-838c-5440c3229851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:11.277549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c41407-3e88-4c8b-838c-5440c3229851 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:11.277835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c41407-3e88-4c8b-838c-5440c3229851 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:11.278108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c41407-3e88-4c8b-838c-5440c3229851 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:11.278846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c41407-3e88-4c8b-838c-5440c3229851 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:11.279150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c41407-3e88-4c8b-838c-5440c3229851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:11.280541 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c41407-3e88-4c8b-838c-5440c3229851 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:11.288692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 543/1081] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:55:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:11.294593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e170f-7141-425c-813e-da075aaca526 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:11.392249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e170f-7141-425c-813e-da075aaca526 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:11.423923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f57e2a-ce75-4ba8-b881-68a23bc22467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:11.424781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f57e2a-ce75-4ba8-b881-68a23bc22467 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:11.425059 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f57e2a-ce75-4ba8-b881-68a23bc22467 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:11.425860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f57e2a-ce75-4ba8-b881-68a23bc22467 None admin] PATH_INFO: `/v3/users/4e3a2f1d31524717bf9cf49331b28e47` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:11.427026 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f57e2a-ce75-4ba8-b881-68a23bc22467 None admin] RBAC: Authorizing `identity:get_user(user_id=4e3a2f1d31524717bf9cf49331b28e47)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:11.428503 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f57e2a-ce75-4ba8-b881-68a23bc22467 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:11.430112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 539/1082] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:11 2022] GET /identity/v3/users/4e3a2f1d31524717bf9cf49331b28e47 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:11.436607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c41407-3e88-4c8b-838c-5440c3229851 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:11.534781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c41407-3e88-4c8b-838c-5440c3229851 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:11.569201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe6179-99ae-49c8-bf07-c53e3de9522f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:11.570059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe6179-99ae-49c8-bf07-c53e3de9522f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:11.570342 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe6179-99ae-49c8-bf07-c53e3de9522f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:11.570619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe6179-99ae-49c8-bf07-c53e3de9522f None admin] PATH_INFO: `/v3/projects/7bcccf0364454757a67b6b769795f701` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:11.571803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe6179-99ae-49c8-bf07-c53e3de9522f None admin] RBAC: Authorizing `identity:get_project(project_id=7bcccf0364454757a67b6b769795f701)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:11.573223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe6179-99ae-49c8-bf07-c53e3de9522f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:11.574915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 544/1083] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:55:11 2022] GET /identity/v3/projects/7bcccf0364454757a67b6b769795f701 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:11.580788 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f57e2a-ce75-4ba8-b881-68a23bc22467 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:11.584571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:11.686345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f57e2a-ce75-4ba8-b881-68a23bc22467 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:11.719402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3245026-a24b-4def-a46e-994424d87da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:11.720290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3245026-a24b-4def-a46e-994424d87da5 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:11.720578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3245026-a24b-4def-a46e-994424d87da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:11.720860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3245026-a24b-4def-a46e-994424d87da5 None admin] PATH_INFO: `/v3/projects/7bcccf0364454757a67b6b769795f701/users/4e3a2f1d31524717bf9cf49331b28e47/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:11.723364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3245026-a24b-4def-a46e-994424d87da5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7bcccf0364454757a67b6b769795f701, user_id=4e3a2f1d31524717bf9cf49331b28e47, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:11.724842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3245026-a24b-4def-a46e-994424d87da5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:11.738566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 540/1084] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:55:11 2022] PUT /identity/v3/projects/7bcccf0364454757a67b6b769795f701/users/4e3a2f1d31524717bf9cf49331b28e47/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:55:12.839483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409e8b9-351a-4c1a-a3bd-32925bf6ef87 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:12.840062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409e8b9-351a-4c1a-a3bd-32925bf6ef87 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:12.840062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409e8b9-351a-4c1a-a3bd-32925bf6ef87 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:12.841420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 545/1085] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:12.846944 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c7944-023d-495c-9517-4bd6808e6eb2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:12.847375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c7944-023d-495c-9517-4bd6808e6eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:12.847546 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c7944-023d-495c-9517-4bd6808e6eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:12.864795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d71c7944-023d-495c-9517-4bd6808e6eb2 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:12.865985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d71c7944-023d-495c-9517-4bd6808e6eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:12.923857 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:12.934058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 541/1086] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:12.958270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409e8b9-351a-4c1a-a3bd-32925bf6ef87 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:12.976776 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:13.061428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409e8b9-351a-4c1a-a3bd-32925bf6ef87 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:13.095066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f594e40-2096-4f98-b712-3aa4285844b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:13.095838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f594e40-2096-4f98-b712-3aa4285844b0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:13.096128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f594e40-2096-4f98-b712-3aa4285844b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:13.096408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f594e40-2096-4f98-b712-3aa4285844b0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:13.106400 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f594e40-2096-4f98-b712-3aa4285844b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-9f594e40-2096-4f98-b712-3aa4285844b0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:55:13.115152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:13.116101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:13.116933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 546/1087] 10.222.0.75 () {56 vars in 1120 bytes} [Thu Jul 14 10:55:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:13.122431 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c7944-023d-495c-9517-4bd6808e6eb2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:13.234176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c7944-023d-495c-9517-4bd6808e6eb2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:13.273757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd55446-1f9b-4737-9918-c42e671e1e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:13.274469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd55446-1f9b-4737-9918-c42e671e1e31 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:13.275020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd55446-1f9b-4737-9918-c42e671e1e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:13.275312 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd55446-1f9b-4737-9918-c42e671e1e31 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:13.276209 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd55446-1f9b-4737-9918-c42e671e1e31 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:13.276523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd55446-1f9b-4737-9918-c42e671e1e31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:13.278378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd55446-1f9b-4737-9918-c42e671e1e31 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:13.287728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 542/1088] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Jul 14 10:55:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:13.293385 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f594e40-2096-4f98-b712-3aa4285844b0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:13.373585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:13.397221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f594e40-2096-4f98-b712-3aa4285844b0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:13.431418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272ff74-9dd4-43e1-b4ea-19b4931336ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:13.432230 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272ff74-9dd4-43e1-b4ea-19b4931336ce None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:13.432520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272ff74-9dd4-43e1-b4ea-19b4931336ce None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:13.432805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272ff74-9dd4-43e1-b4ea-19b4931336ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:13.444222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272ff74-9dd4-43e1-b4ea-19b4931336ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:13.445687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272ff74-9dd4-43e1-b4ea-19b4931336ce None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-1272ff74-9dd4-43e1-b4ea-19b4931336ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:55:13.463062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:13.464167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:13.465132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:55:13.465132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:55:13.465132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:13.465132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:13.465132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 547/1089] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:55:13.470006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd55446-1f9b-4737-9918-c42e671e1e31 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:13.525932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:13.570202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd55446-1f9b-4737-9918-c42e671e1e31 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:13.603484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf305a-9c4c-4206-9da1-2d755acb064d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:13.604159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf305a-9c4c-4206-9da1-2d755acb064d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:13.604443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf305a-9c4c-4206-9da1-2d755acb064d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:13.604720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf305a-9c4c-4206-9da1-2d755acb064d None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:13.605441 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf305a-9c4c-4206-9da1-2d755acb064d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:13.605746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf305a-9c4c-4206-9da1-2d755acb064d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:13.607125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf305a-9c4c-4206-9da1-2d755acb064d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:13.619254 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 543/1090] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:13.624875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272ff74-9dd4-43e1-b4ea-19b4931336ce None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:13.727765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272ff74-9dd4-43e1-b4ea-19b4931336ce None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:13.754066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:13.754960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0756de-6da0-4edf-bffd-37945fbd3ce6 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:13.755233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0756de-6da0-4edf-bffd-37945fbd3ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:13.755550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0756de-6da0-4edf-bffd-37945fbd3ce6 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:13.756944 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 544/1091] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:13.762128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587a8ed-3509-4592-a735-c9b8e9597cbb None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:13.762419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587a8ed-3509-4592-a735-c9b8e9597cbb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:13.762696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587a8ed-3509-4592-a735-c9b8e9597cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:13.763489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccec8d6-612c-4356-a4f8-56a21334376a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:13.764295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccec8d6-612c-4356-a4f8-56a21334376a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:13.764585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccec8d6-612c-4356-a4f8-56a21334376a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:13.764864 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccec8d6-612c-4356-a4f8-56a21334376a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:13.773502 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccec8d6-612c-4356-a4f8-56a21334376a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:13.774967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccec8d6-612c-4356-a4f8-56a21334376a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:13.779645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a587a8ed-3509-4592-a735-c9b8e9597cbb None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:13.780850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a587a8ed-3509-4592-a735-c9b8e9597cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-5ccec8d6-612c-4356-a4f8-56a21334376a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:55:13.789842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:13.790818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:13.791803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:13.791803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:13.791803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:55:13.791803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:55:13.791803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:55:13.791803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:55:13.791803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:55:13.791803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:55:13.791803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:55:13.791803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:13.791803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 548/1092] 10.222.0.75 () {56 vars in 1093 bytes} [Thu Jul 14 10:55:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:13.796726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccec8d6-612c-4356-a4f8-56a21334376a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:13.855572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 545/1093] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:55:13.876298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587a8ed-3509-4592-a735-c9b8e9597cbb None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:13.900210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccec8d6-612c-4356-a4f8-56a21334376a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:13.922625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:13.935413 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6d9eb-ca75-4173-b3d4-bbd98bc16ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:13.936099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6d9eb-ca75-4173-b3d4-bbd98bc16ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:13.936384 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6d9eb-ca75-4173-b3d4-bbd98bc16ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:13.936667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6d9eb-ca75-4173-b3d4-bbd98bc16ac9 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:13.937501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6d9eb-ca75-4173-b3d4-bbd98bc16ac9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:13.937805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6d9eb-ca75-4173-b3d4-bbd98bc16ac9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:13.939244 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6d9eb-ca75-4173-b3d4-bbd98bc16ac9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:13.957556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 549/1094] 10.222.0.75 () {56 vars in 1133 bytes} [Thu Jul 14 10:55:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:13.963053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6d9eb-ca75-4173-b3d4-bbd98bc16ac9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:13.977018 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587a8ed-3509-4592-a735-c9b8e9597cbb None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:14.010401 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:14.011531 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aec61-75a7-412e-8996-04add7095212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:14.012697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aec61-75a7-412e-8996-04add7095212 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:14.013030 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aec61-75a7-412e-8996-04add7095212 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:14.013489 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aec61-75a7-412e-8996-04add7095212 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:14.024547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3aec61-75a7-412e-8996-04add7095212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-2f3aec61-75a7-412e-8996-04add7095212 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:55:14.032347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:14.033361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:14.034211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 546/1095] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:55:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:55:14.039525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aec61-75a7-412e-8996-04add7095212 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:14.067302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6d9eb-ca75-4173-b3d4-bbd98bc16ac9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:14.101367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b8c27-904f-4691-acab-0284a9d83f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:14.102014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b8c27-904f-4691-acab-0284a9d83f79 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:14.102298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b8c27-904f-4691-acab-0284a9d83f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:14.102574 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b8c27-904f-4691-acab-0284a9d83f79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:14.140750 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aec61-75a7-412e-8996-04add7095212 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:14.148698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:14.169735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b8c27-904f-4691-acab-0284a9d83f79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:14.171182 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b8c27-904f-4691-acab-0284a9d83f79 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:14.176093 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059142e-60d5-4486-a6f2-f3d2f02b034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:14.176767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059142e-60d5-4486-a6f2-f3d2f02b034a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:14.177055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059142e-60d5-4486-a6f2-f3d2f02b034a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:14.177336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059142e-60d5-4486-a6f2-f3d2f02b034a None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:14.178082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059142e-60d5-4486-a6f2-f3d2f02b034a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:14.178391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059142e-60d5-4486-a6f2-f3d2f02b034a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:14.179819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059142e-60d5-4486-a6f2-f3d2f02b034a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:14.188515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 547/1096] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:55:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:14.195110 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059142e-60d5-4486-a6f2-f3d2f02b034a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:14.220293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:14.276628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 550/1097] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:55:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:55:14.282331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b8c27-904f-4691-acab-0284a9d83f79 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:14.299078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059142e-60d5-4486-a6f2-f3d2f02b034a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:14.337630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f3b03-c928-491c-9c6c-ce7f3d59dab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:14.338505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f3b03-c928-491c-9c6c-ce7f3d59dab4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:14.338795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f3b03-c928-491c-9c6c-ce7f3d59dab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:14.339071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f3b03-c928-491c-9c6c-ce7f3d59dab4 None admin] PATH_INFO: `/v3/users/4e3a2f1d31524717bf9cf49331b28e47` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:14.340324 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f3b03-c928-491c-9c6c-ce7f3d59dab4 None admin] RBAC: Authorizing `identity:get_user(user_id=4e3a2f1d31524717bf9cf49331b28e47)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:14.341772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f3b03-c928-491c-9c6c-ce7f3d59dab4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:14.343432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 548/1098] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:14 2022] GET /identity/v3/users/4e3a2f1d31524717bf9cf49331b28e47 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:14.349240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f3b03-c928-491c-9c6c-ce7f3d59dab4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:14.382295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b8c27-904f-4691-acab-0284a9d83f79 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:14.385055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:14.417861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84c532-f2e1-4151-9ad9-595ec651f21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:14.418577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84c532-f2e1-4151-9ad9-595ec651f21c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:14.418863 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84c532-f2e1-4151-9ad9-595ec651f21c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:14.419138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84c532-f2e1-4151-9ad9-595ec651f21c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:14.430060 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84c532-f2e1-4151-9ad9-595ec651f21c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:14.431581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84c532-f2e1-4151-9ad9-595ec651f21c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:14.439022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-ee84c532-f2e1-4151-9ad9-595ec651f21c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:55:14.451765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:14.452762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:14.453665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:55:14.453665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:55:14.453665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:55:14.453665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:14.453895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f3b03-c928-491c-9c6c-ce7f3d59dab4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:14.453979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 551/1099] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:14.458769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84c532-f2e1-4151-9ad9-595ec651f21c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:14.486867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719968-13f0-4550-96c2-7c721ba315ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:14.488120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719968-13f0-4550-96c2-7c721ba315ca None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:14.488405 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719968-13f0-4550-96c2-7c721ba315ca None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:14.488678 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719968-13f0-4550-96c2-7c721ba315ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:14.555801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a719968-13f0-4550-96c2-7c721ba315ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:14.557273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a719968-13f0-4550-96c2-7c721ba315ca None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:14.561275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84c532-f2e1-4151-9ad9-595ec651f21c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:14.590991 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:14.597368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b12ac-bfea-4ec3-a866-b0bdc842bb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:14.598037 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b12ac-bfea-4ec3-a866-b0bdc842bb6f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:14.598322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b12ac-bfea-4ec3-a866-b0bdc842bb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:14.598607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b12ac-bfea-4ec3-a866-b0bdc842bb6f None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:14.599328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b12ac-bfea-4ec3-a866-b0bdc842bb6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:14.599695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b12ac-bfea-4ec3-a866-b0bdc842bb6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:14.601072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b12ac-bfea-4ec3-a866-b0bdc842bb6f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:14.613272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 552/1100] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:14.618815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b12ac-bfea-4ec3-a866-b0bdc842bb6f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:14.658448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 549/1101] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:55:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:55:14.664479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719968-13f0-4550-96c2-7c721ba315ca None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:14.678095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:14.721365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b12ac-bfea-4ec3-a866-b0bdc842bb6f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:14.755601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58bbf8-ca5d-4a3c-8db4-5873a5acf068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:14.756400 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58bbf8-ca5d-4a3c-8db4-5873a5acf068 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:14.756695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58bbf8-ca5d-4a3c-8db4-5873a5acf068 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:14.756979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58bbf8-ca5d-4a3c-8db4-5873a5acf068 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:14.762418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719968-13f0-4550-96c2-7c721ba315ca None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:14.763890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58bbf8-ca5d-4a3c-8db4-5873a5acf068 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:14.765343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58bbf8-ca5d-4a3c-8db4-5873a5acf068 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-ac58bbf8-ca5d-4a3c-8db4-5873a5acf068 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:55:14.779499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:14.780567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:14.781623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:55:14.781623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:55:14.781623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:55:14.781623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:55:14.781623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:55:14.781623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:55:14.781623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:55:14.781623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:14.781623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 553/1102] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:55:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:14.787127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58bbf8-ca5d-4a3c-8db4-5873a5acf068 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:14.796136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22ba0f-f818-4830-a358-1e127f1e2dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:14.796980 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22ba0f-f818-4830-a358-1e127f1e2dfd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:14.797262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22ba0f-f818-4830-a358-1e127f1e2dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:14.797530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22ba0f-f818-4830-a358-1e127f1e2dfd None admin] PATH_INFO: `/v3/projects/7bcccf0364454757a67b6b769795f701` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:14.798695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22ba0f-f818-4830-a358-1e127f1e2dfd None admin] RBAC: Authorizing `identity:get_project(project_id=7bcccf0364454757a67b6b769795f701)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:14.800704 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22ba0f-f818-4830-a358-1e127f1e2dfd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:14.802504 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 550/1103] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:55:14 2022] GET /identity/v3/projects/7bcccf0364454757a67b6b769795f701 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:14.808465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22ba0f-f818-4830-a358-1e127f1e2dfd None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:14.811930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:14.892138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58bbf8-ca5d-4a3c-8db4-5873a5acf068 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:14.908063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22ba0f-f818-4830-a358-1e127f1e2dfd None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:14.909014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:14.928844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455b4f4-db22-4186-82a1-1877207d98cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:14.929618 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455b4f4-db22-4186-82a1-1877207d98cf None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:14.930250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455b4f4-db22-4186-82a1-1877207d98cf None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:14.930824 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455b4f4-db22-4186-82a1-1877207d98cf None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:14.931552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455b4f4-db22-4186-82a1-1877207d98cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:14.931878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455b4f4-db22-4186-82a1-1877207d98cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:14.933597 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455b4f4-db22-4186-82a1-1877207d98cf None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:14.945835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 554/1104] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:55:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:14.946746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38578d-bd5f-4756-af54-58cd759261b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:14.947441 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38578d-bd5f-4756-af54-58cd759261b2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:14.947728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38578d-bd5f-4756-af54-58cd759261b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:14.948004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38578d-bd5f-4756-af54-58cd759261b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:14.948849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38578d-bd5f-4756-af54-58cd759261b2 None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=7bcccf0364454757a67b6b769795f701, user.id=4e3a2f1d31524717bf9cf49331b28e47) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:14.949162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38578d-bd5f-4756-af54-58cd759261b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:14.950636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38578d-bd5f-4756-af54-58cd759261b2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:14.951910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455b4f4-db22-4186-82a1-1877207d98cf None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:14.957372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 551/1105] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:55:14 2022] GET /identity/v3/role_assignments?user.id=4e3a2f1d31524717bf9cf49331b28e47&scope.project.id=7bcccf0364454757a67b6b769795f701&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:15.065894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455b4f4-db22-4186-82a1-1877207d98cf None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:15.102218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825f513-e235-4804-918c-e0f2682e5f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:15.102880 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825f513-e235-4804-918c-e0f2682e5f8a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:15.103161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825f513-e235-4804-918c-e0f2682e5f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:15.103462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825f513-e235-4804-918c-e0f2682e5f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:15.104298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825f513-e235-4804-918c-e0f2682e5f8a None admin] RBAC: Adding query filter params (role.id=4d8a62dacbc5492b9fa4b11c57673134, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=7e572bf785284a8693cf386ced004524) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:15.104612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825f513-e235-4804-918c-e0f2682e5f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:15.106048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825f513-e235-4804-918c-e0f2682e5f8a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:15.112513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 555/1106] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:55:14 2022] GET /identity/v3/role_assignments?user.id=7e572bf785284a8693cf386ced004524&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=4d8a62dacbc5492b9fa4b11c57673134 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:17.043704 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32cfe1-d7e1-4f49-81b0-6457cbd1723f None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:17.044262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32cfe1-d7e1-4f49-81b0-6457cbd1723f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:17.044262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32cfe1-d7e1-4f49-81b0-6457cbd1723f None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:17.045591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 552/1107] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:17.050757 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4edd79-3730-40ab-b59b-7092c4f19bab None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:17.051144 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4edd79-3730-40ab-b59b-7092c4f19bab None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:17.051321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4edd79-3730-40ab-b59b-7092c4f19bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:17.068489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9f4edd79-3730-40ab-b59b-7092c4f19bab None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:17.069672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9f4edd79-3730-40ab-b59b-7092c4f19bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:17.103926 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:17.136291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 556/1108] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:17.160886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32cfe1-d7e1-4f49-81b0-6457cbd1723f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:17.228556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:17.265234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32cfe1-d7e1-4f49-81b0-6457cbd1723f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:17.301117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda459c-66a9-4bbe-b519-9321999ce775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:17.301826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda459c-66a9-4bbe-b519-9321999ce775 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:17.302108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda459c-66a9-4bbe-b519-9321999ce775 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:17.302389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda459c-66a9-4bbe-b519-9321999ce775 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:17.303596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda459c-66a9-4bbe-b519-9321999ce775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:17.305049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda459c-66a9-4bbe-b519-9321999ce775 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:17.306686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 553/1109] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:17.312763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4edd79-3730-40ab-b59b-7092c4f19bab None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:17.419956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4edd79-3730-40ab-b59b-7092c4f19bab None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:17.455070 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e522e-dd96-4fa4-a5da-f192f46fb253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:17.455773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e522e-dd96-4fa4-a5da-f192f46fb253 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:17.456058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e522e-dd96-4fa4-a5da-f192f46fb253 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:17.456335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e522e-dd96-4fa4-a5da-f192f46fb253 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:17.457057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e522e-dd96-4fa4-a5da-f192f46fb253 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:17.458475 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e522e-dd96-4fa4-a5da-f192f46fb253 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:17.477725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 557/1110] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:55:17 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:55:19.529824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25125634-82ae-4345-af13-22859c85ff2f None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:19.530342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25125634-82ae-4345-af13-22859c85ff2f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:19.530342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25125634-82ae-4345-af13-22859c85ff2f None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:19.531700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 554/1111] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:19.537900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfada5f6-6b47-4418-b6f5-9eb620acf167 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:19.538374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfada5f6-6b47-4418-b6f5-9eb620acf167 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:19.538836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfada5f6-6b47-4418-b6f5-9eb620acf167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:19.557666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-dfada5f6-6b47-4418-b6f5-9eb620acf167 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:19.559120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-dfada5f6-6b47-4418-b6f5-9eb620acf167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:19.582306 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:19.626929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 558/1112] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:19.651931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25125634-82ae-4345-af13-22859c85ff2f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:19.752099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25125634-82ae-4345-af13-22859c85ff2f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:19.785477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072fb04-a370-4b68-949d-36b2b23c7aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:19.786253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072fb04-a370-4b68-949d-36b2b23c7aec None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:19.786540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072fb04-a370-4b68-949d-36b2b23c7aec None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:19.786828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072fb04-a370-4b68-949d-36b2b23c7aec None admin] PATH_INFO: `/v3/roles/ff3de46e4867421f990563b41a305d4d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:19.788322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072fb04-a370-4b68-949d-36b2b23c7aec None admin] RBAC: Authorizing `identity:get_role(role_id=ff3de46e4867421f990563b41a305d4d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:19.789700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072fb04-a370-4b68-949d-36b2b23c7aec None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:19.791071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 555/1113] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:19 2022] GET /identity/v3/roles/ff3de46e4867421f990563b41a305d4d => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:19.796932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfada5f6-6b47-4418-b6f5-9eb620acf167 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:19.893713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfada5f6-6b47-4418-b6f5-9eb620acf167 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:19.926736 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb2038-9bc2-4bc6-876f-5d774d43150d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:19.927635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb2038-9bc2-4bc6-876f-5d774d43150d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:19.927920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb2038-9bc2-4bc6-876f-5d774d43150d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:19.928198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb2038-9bc2-4bc6-876f-5d774d43150d None admin] PATH_INFO: `/v3/users/c2ecfd9f768e40bcb115775671e66c5d` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:19.941853 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb2038-9bc2-4bc6-876f-5d774d43150d None admin] RBAC: Authorizing `identity:get_user(user_id=c2ecfd9f768e40bcb115775671e66c5d)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:19.943299 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb2038-9bc2-4bc6-876f-5d774d43150d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:19.944992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 559/1114] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:19 2022] GET /identity/v3/users/c2ecfd9f768e40bcb115775671e66c5d => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:19.950799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072fb04-a370-4b68-949d-36b2b23c7aec None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:19.954588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:20.053248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072fb04-a370-4b68-949d-36b2b23c7aec None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:20.085208 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf330e9e-05fb-4fa9-9514-5803291453b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:20.085853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf330e9e-05fb-4fa9-9514-5803291453b2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:20.086135 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf330e9e-05fb-4fa9-9514-5803291453b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:20.086411 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf330e9e-05fb-4fa9-9514-5803291453b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:20.150832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf330e9e-05fb-4fa9-9514-5803291453b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:20.152297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf330e9e-05fb-4fa9-9514-5803291453b2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:20.154801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:20.248224 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 556/1115] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:55:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:55:20.254138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb2038-9bc2-4bc6-876f-5d774d43150d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:20.272091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:20.353459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb2038-9bc2-4bc6-876f-5d774d43150d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:20.386016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d5175-30be-4590-a1e4-4025289b2494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:20.386884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d5175-30be-4590-a1e4-4025289b2494 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:20.387171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d5175-30be-4590-a1e4-4025289b2494 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:20.387487 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d5175-30be-4590-a1e4-4025289b2494 None admin] PATH_INFO: `/v3/projects/7bcccf0364454757a67b6b769795f701` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:20.388659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d5175-30be-4590-a1e4-4025289b2494 None admin] RBAC: Authorizing `identity:get_project(project_id=7bcccf0364454757a67b6b769795f701)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:20.390098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d5175-30be-4590-a1e4-4025289b2494 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:20.391845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 560/1116] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:55:20 2022] GET /identity/v3/projects/7bcccf0364454757a67b6b769795f701 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:20.398538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf330e9e-05fb-4fa9-9514-5803291453b2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:20.495211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf330e9e-05fb-4fa9-9514-5803291453b2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:20.497527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:20.529882 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ba9db-10eb-4815-aaff-59815fd90894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:20.530539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ba9db-10eb-4815-aaff-59815fd90894 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:20.530807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ba9db-10eb-4815-aaff-59815fd90894 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:20.531083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ba9db-10eb-4815-aaff-59815fd90894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:20.531938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ba9db-10eb-4815-aaff-59815fd90894 None admin] RBAC: Adding query filter params (role.id=ff3de46e4867421f990563b41a305d4d, scope.project.id=7bcccf0364454757a67b6b769795f701, user.id=c2ecfd9f768e40bcb115775671e66c5d) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:20.532248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ba9db-10eb-4815-aaff-59815fd90894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:20.533669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ba9db-10eb-4815-aaff-59815fd90894 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:20.539701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 557/1117] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:55:20 2022] GET /identity/v3/role_assignments?user.id=c2ecfd9f768e40bcb115775671e66c5d&scope.project.id=7bcccf0364454757a67b6b769795f701&role.id=ff3de46e4867421f990563b41a305d4d => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:22.538271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39023e8f-3639-49b4-acaa-6f59852e5cc2 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:22.538791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39023e8f-3639-49b4-acaa-6f59852e5cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:22.538791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39023e8f-3639-49b4-acaa-6f59852e5cc2 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:22.540211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 561/1118] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:22.545454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808771a-34a9-4518-b18e-cdab2583158d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:22.545849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808771a-34a9-4518-b18e-cdab2583158d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:22.546023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808771a-34a9-4518-b18e-cdab2583158d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:22.563468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6808771a-34a9-4518-b18e-cdab2583158d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:22.564657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6808771a-34a9-4518-b18e-cdab2583158d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:22.628302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 558/1119] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:22.651288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39023e8f-3639-49b4-acaa-6f59852e5cc2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:22.736291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:22.754826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39023e8f-3639-49b4-acaa-6f59852e5cc2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:22.788047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31955a6b-c91d-4abc-8d46-7919a8fd5529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:22.788944 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31955a6b-c91d-4abc-8d46-7919a8fd5529 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:22.789176 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31955a6b-c91d-4abc-8d46-7919a8fd5529 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:22.789452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31955a6b-c91d-4abc-8d46-7919a8fd5529 None admin] PATH_INFO: `/v3/roles/ff3de46e4867421f990563b41a305d4d` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:22.790900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31955a6b-c91d-4abc-8d46-7919a8fd5529 None admin] RBAC: Authorizing `identity:get_role(role_id=ff3de46e4867421f990563b41a305d4d)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:22.792305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31955a6b-c91d-4abc-8d46-7919a8fd5529 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:22.793669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 562/1120] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:22 2022] GET /identity/v3/roles/ff3de46e4867421f990563b41a305d4d => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:22.799578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808771a-34a9-4518-b18e-cdab2583158d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:22.895140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:22.899633 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808771a-34a9-4518-b18e-cdab2583158d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:22.933206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedefaea-877f-42c3-9de7-01e7b7a17c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:22.934082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedefaea-877f-42c3-9de7-01e7b7a17c46 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:22.934360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedefaea-877f-42c3-9de7-01e7b7a17c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:22.934630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedefaea-877f-42c3-9de7-01e7b7a17c46 None admin] PATH_INFO: `/v3/users/c2ecfd9f768e40bcb115775671e66c5d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:22.935822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedefaea-877f-42c3-9de7-01e7b7a17c46 None admin] RBAC: Authorizing `identity:get_user(user_id=c2ecfd9f768e40bcb115775671e66c5d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:22.937255 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedefaea-877f-42c3-9de7-01e7b7a17c46 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:22.938865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 559/1121] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:22 2022] GET /identity/v3/users/c2ecfd9f768e40bcb115775671e66c5d => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:22.945313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31955a6b-c91d-4abc-8d46-7919a8fd5529 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:23.045573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31955a6b-c91d-4abc-8d46-7919a8fd5529 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:23.078561 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1da9c-4718-427c-9466-0afe68ebeb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:23.079449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1da9c-4718-427c-9466-0afe68ebeb4f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:23.079740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1da9c-4718-427c-9466-0afe68ebeb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:23.080023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1da9c-4718-427c-9466-0afe68ebeb4f None admin] PATH_INFO: `/v3/projects/7bcccf0364454757a67b6b769795f701` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:23.081255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1da9c-4718-427c-9466-0afe68ebeb4f None admin] RBAC: Authorizing `identity:get_project(project_id=7bcccf0364454757a67b6b769795f701)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:23.082702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1da9c-4718-427c-9466-0afe68ebeb4f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:23.084439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 563/1122] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:55:22 2022] GET /identity/v3/projects/7bcccf0364454757a67b6b769795f701 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:23.090386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedefaea-877f-42c3-9de7-01e7b7a17c46 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:23.188639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedefaea-877f-42c3-9de7-01e7b7a17c46 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:23.222443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb0060-70c2-4550-a1b1-ba2d6b2fe03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:23.223327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb0060-70c2-4550-a1b1-ba2d6b2fe03c None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:23.223657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb0060-70c2-4550-a1b1-ba2d6b2fe03c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:23.223938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb0060-70c2-4550-a1b1-ba2d6b2fe03c None admin] PATH_INFO: `/v3/projects/7bcccf0364454757a67b6b769795f701/users/c2ecfd9f768e40bcb115775671e66c5d/roles/ff3de46e4867421f990563b41a305d4d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:23.226143 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb0060-70c2-4550-a1b1-ba2d6b2fe03c None admin] RBAC: Authorizing `identity:create_grant(project_id=7bcccf0364454757a67b6b769795f701, user_id=c2ecfd9f768e40bcb115775671e66c5d, role_id=ff3de46e4867421f990563b41a305d4d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:23.227621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb0060-70c2-4550-a1b1-ba2d6b2fe03c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:23.240832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 560/1123] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:55:23 2022] PUT /identity/v3/projects/7bcccf0364454757a67b6b769795f701/users/c2ecfd9f768e40bcb115775671e66c5d/roles/ff3de46e4867421f990563b41a305d4d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:55:25.256201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782fd33-8f6b-488f-9575-b0a0a404447c None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:25.256741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782fd33-8f6b-488f-9575-b0a0a404447c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:25.256741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782fd33-8f6b-488f-9575-b0a0a404447c None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:25.258127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 564/1124] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:25.263427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182739c-c3f8-445d-989d-ee397f438f28 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:25.263867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182739c-c3f8-445d-989d-ee397f438f28 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:25.264027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182739c-c3f8-445d-989d-ee397f438f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:25.280884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d182739c-c3f8-445d-989d-ee397f438f28 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:25.282071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d182739c-c3f8-445d-989d-ee397f438f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:25.301344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:25.352366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 561/1125] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:25.375921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782fd33-8f6b-488f-9575-b0a0a404447c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:25.402173 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:25.480185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782fd33-8f6b-488f-9575-b0a0a404447c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:25.513227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83200a-5b15-4cf4-aa0d-fb6442dead01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:25.514090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83200a-5b15-4cf4-aa0d-fb6442dead01 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:25.514376 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83200a-5b15-4cf4-aa0d-fb6442dead01 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:25.514660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83200a-5b15-4cf4-aa0d-fb6442dead01 None admin] PATH_INFO: `/v3/roles/ff3de46e4867421f990563b41a305d4d` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:25.516151 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83200a-5b15-4cf4-aa0d-fb6442dead01 None admin] RBAC: Authorizing `identity:get_role(role_id=ff3de46e4867421f990563b41a305d4d)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:25.517522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83200a-5b15-4cf4-aa0d-fb6442dead01 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:25.519495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 565/1126] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:25 2022] GET /identity/v3/roles/ff3de46e4867421f990563b41a305d4d => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:25.525407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182739c-c3f8-445d-989d-ee397f438f28 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:25.620688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182739c-c3f8-445d-989d-ee397f438f28 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:25.653191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bc623-251c-4720-afc0-d883e1dc96a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:25.654060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bc623-251c-4720-afc0-d883e1dc96a1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:25.654342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bc623-251c-4720-afc0-d883e1dc96a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:25.654615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bc623-251c-4720-afc0-d883e1dc96a1 None admin] PATH_INFO: `/v3/users/c2ecfd9f768e40bcb115775671e66c5d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:25.655843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bc623-251c-4720-afc0-d883e1dc96a1 None admin] RBAC: Authorizing `identity:get_user(user_id=c2ecfd9f768e40bcb115775671e66c5d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:25.657289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bc623-251c-4720-afc0-d883e1dc96a1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:25.658921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 562/1127] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:25 2022] GET /identity/v3/users/c2ecfd9f768e40bcb115775671e66c5d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:25.664892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83200a-5b15-4cf4-aa0d-fb6442dead01 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:25.751543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:25.764759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83200a-5b15-4cf4-aa0d-fb6442dead01 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:25.797398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4e983-4db5-401c-a611-6a35b291331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:25.798042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4e983-4db5-401c-a611-6a35b291331c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:25.798324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4e983-4db5-401c-a611-6a35b291331c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:25.798600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4e983-4db5-401c-a611-6a35b291331c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:25.863137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4e983-4db5-401c-a611-6a35b291331c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:25.864600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4e983-4db5-401c-a611-6a35b291331c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:25.946448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:25.959290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 566/1128] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:55:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:55:25.965187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bc623-251c-4720-afc0-d883e1dc96a1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:25.990585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:26.063373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bc623-251c-4720-afc0-d883e1dc96a1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:26.095637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf94b9a-f781-4637-a0b0-345faf3d9d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:26.096514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf94b9a-f781-4637-a0b0-345faf3d9d80 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:26.097327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf94b9a-f781-4637-a0b0-345faf3d9d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:26.097606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf94b9a-f781-4637-a0b0-345faf3d9d80 None admin] PATH_INFO: `/v3/projects/7bcccf0364454757a67b6b769795f701` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:26.098855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf94b9a-f781-4637-a0b0-345faf3d9d80 None admin] RBAC: Authorizing `identity:get_project(project_id=7bcccf0364454757a67b6b769795f701)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:26.100348 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf94b9a-f781-4637-a0b0-345faf3d9d80 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:26.102074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 563/1129] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:55:25 2022] GET /identity/v3/projects/7bcccf0364454757a67b6b769795f701 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:26.108312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4e983-4db5-401c-a611-6a35b291331c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:26.206700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4e983-4db5-401c-a611-6a35b291331c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:26.241344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa74bf3-88e9-4f91-ab52-f72b97b09c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:26.242000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa74bf3-88e9-4f91-ab52-f72b97b09c0e None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:26.242288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa74bf3-88e9-4f91-ab52-f72b97b09c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:26.242572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa74bf3-88e9-4f91-ab52-f72b97b09c0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:26.243695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa74bf3-88e9-4f91-ab52-f72b97b09c0e None admin] RBAC: Adding query filter params (role.id=ff3de46e4867421f990563b41a305d4d, scope.project.id=7bcccf0364454757a67b6b769795f701, user.id=c2ecfd9f768e40bcb115775671e66c5d) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:26.243803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa74bf3-88e9-4f91-ab52-f72b97b09c0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:26.245167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa74bf3-88e9-4f91-ab52-f72b97b09c0e None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:26.250925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 567/1130] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:55:26 2022] GET /identity/v3/role_assignments?user.id=c2ecfd9f768e40bcb115775671e66c5d&scope.project.id=7bcccf0364454757a67b6b769795f701&role.id=ff3de46e4867421f990563b41a305d4d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:28.293810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58f342-b77e-4462-9d3e-686618fa36aa None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:28.294361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58f342-b77e-4462-9d3e-686618fa36aa None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:28.294454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58f342-b77e-4462-9d3e-686618fa36aa None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:28.295818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 564/1131] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:28.300852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba971fcc-7c07-451d-85c9-e22c8c40bc45 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:28.301291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba971fcc-7c07-451d-85c9-e22c8c40bc45 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:28.301416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba971fcc-7c07-451d-85c9-e22c8c40bc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:28.319516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ba971fcc-7c07-451d-85c9-e22c8c40bc45 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:28.320728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ba971fcc-7c07-451d-85c9-e22c8c40bc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:28.366979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:28.387467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 568/1132] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:28.411157 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58f342-b77e-4462-9d3e-686618fa36aa None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:28.501151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:28.511614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58f342-b77e-4462-9d3e-686618fa36aa None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:28.544871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b18617-043b-4c12-b410-0a4002b4d3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:28.545666 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b18617-043b-4c12-b410-0a4002b4d3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:28.545953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b18617-043b-4c12-b410-0a4002b4d3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:28.546233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b18617-043b-4c12-b410-0a4002b4d3c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:28.547423 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b18617-043b-4c12-b410-0a4002b4d3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:28.548869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b18617-043b-4c12-b410-0a4002b4d3c9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:28.550499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 565/1133] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:28.556444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba971fcc-7c07-451d-85c9-e22c8c40bc45 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:28.656961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba971fcc-7c07-451d-85c9-e22c8c40bc45 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:28.691655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57f08f-78fa-468c-8487-087b7e0bed5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:28.692320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57f08f-78fa-468c-8487-087b7e0bed5b None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:28.692609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57f08f-78fa-468c-8487-087b7e0bed5b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:28.692891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57f08f-78fa-468c-8487-087b7e0bed5b None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:28.693604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57f08f-78fa-468c-8487-087b7e0bed5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:28.695040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57f08f-78fa-468c-8487-087b7e0bed5b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:28.719799 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 569/1134] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:55:28 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:55:30.772755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e2c35-3b34-4050-97d9-77fd7100f5b9 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:30.773321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e2c35-3b34-4050-97d9-77fd7100f5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:30.773321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e2c35-3b34-4050-97d9-77fd7100f5b9 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:30.774644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 566/1135] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:30.779724 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611f631-3d35-4cfc-a802-7d4bc7ccc46e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:30.780134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611f631-3d35-4cfc-a802-7d4bc7ccc46e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:30.780298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611f631-3d35-4cfc-a802-7d4bc7ccc46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:30.796672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-4611f631-3d35-4cfc-a802-7d4bc7ccc46e None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:30.797864 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4611f631-3d35-4cfc-a802-7d4bc7ccc46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:30.824332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:30.865104 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 570/1136] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:55:30.889523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e2c35-3b34-4050-97d9-77fd7100f5b9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:30.985337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e2c35-3b34-4050-97d9-77fd7100f5b9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:31.018993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139781b0-74cc-4c78-b8ee-60d72621d7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:31.019799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139781b0-74cc-4c78-b8ee-60d72621d7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:31.020083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139781b0-74cc-4c78-b8ee-60d72621d7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:31.020359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139781b0-74cc-4c78-b8ee-60d72621d7c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:31.021527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139781b0-74cc-4c78-b8ee-60d72621d7c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:31.023004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139781b0-74cc-4c78-b8ee-60d72621d7c2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:31.024574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 567/1137] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:55:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:31.030307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611f631-3d35-4cfc-a802-7d4bc7ccc46e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:31.126693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611f631-3d35-4cfc-a802-7d4bc7ccc46e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:31.159688 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17ec73-ac45-452d-ab8b-5bc0bff02dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:31.160356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17ec73-ac45-452d-ab8b-5bc0bff02dea None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:31.160643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17ec73-ac45-452d-ab8b-5bc0bff02dea None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:31.160925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17ec73-ac45-452d-ab8b-5bc0bff02dea None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:31.161637 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17ec73-ac45-452d-ab8b-5bc0bff02dea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:31.163067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17ec73-ac45-452d-ab8b-5bc0bff02dea None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:31.182918 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 571/1138] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:55:31 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:55:33.241917 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a8ad-8590-4656-b044-70603fea50ea None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:33.242427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a8ad-8590-4656-b044-70603fea50ea None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:33.242538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a8ad-8590-4656-b044-70603fea50ea None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:33.243864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 568/1139] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:33.249134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de05d5f-1c55-4d48-9cca-dbaaa477914c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:33.249556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de05d5f-1c55-4d48-9cca-dbaaa477914c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:33.249701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de05d5f-1c55-4d48-9cca-dbaaa477914c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:33.267276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0de05d5f-1c55-4d48-9cca-dbaaa477914c None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:33.268529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0de05d5f-1c55-4d48-9cca-dbaaa477914c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:33.283207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:33.337813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 572/1140] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:33.360966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858a8ad-8590-4656-b044-70603fea50ea None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:33.389691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:33.462212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858a8ad-8590-4656-b044-70603fea50ea None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:33.495189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f45ef-8269-4a3a-ad7c-d4d467bffe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:33.496008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f45ef-8269-4a3a-ad7c-d4d467bffe8c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:33.496290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f45ef-8269-4a3a-ad7c-d4d467bffe8c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:33.496564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f45ef-8269-4a3a-ad7c-d4d467bffe8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:33.506283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f45ef-8269-4a3a-ad7c-d4d467bffe8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-2b0f45ef-8269-4a3a-ad7c-d4d467bffe8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:55:33.513845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:33.515049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:33.516530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 569/1141] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:55:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:33.521201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de05d5f-1c55-4d48-9cca-dbaaa477914c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:33.618803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de05d5f-1c55-4d48-9cca-dbaaa477914c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:33.651431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a4443-0bda-4f9a-9248-acd59d1efca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:33.652100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a4443-0bda-4f9a-9248-acd59d1efca9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:33.652385 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a4443-0bda-4f9a-9248-acd59d1efca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:33.652664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a4443-0bda-4f9a-9248-acd59d1efca9 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:33.653404 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a4443-0bda-4f9a-9248-acd59d1efca9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:33.653712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a4443-0bda-4f9a-9248-acd59d1efca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:33.655088 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a4443-0bda-4f9a-9248-acd59d1efca9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:33.663600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 573/1142] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:55:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:33.669349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f45ef-8269-4a3a-ad7c-d4d467bffe8c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:33.754597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:33.766257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f45ef-8269-4a3a-ad7c-d4d467bffe8c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:33.798475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace25ecd-df79-41f4-aeef-a46ea2a132f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:33.799337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace25ecd-df79-41f4-aeef-a46ea2a132f4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:33.800204 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace25ecd-df79-41f4-aeef-a46ea2a132f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:33.800481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace25ecd-df79-41f4-aeef-a46ea2a132f4 None admin] PATH_INFO: `/v3/users/69b2f1258aad4705a81520417ab6c864` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:33.813445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace25ecd-df79-41f4-aeef-a46ea2a132f4 None admin] RBAC: Authorizing `identity:get_user(user_id=69b2f1258aad4705a81520417ab6c864)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:33.814889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace25ecd-df79-41f4-aeef-a46ea2a132f4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:33.816572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 570/1143] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:33 2022] GET /identity/v3/users/69b2f1258aad4705a81520417ab6c864 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:33.822341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a4443-0bda-4f9a-9248-acd59d1efca9 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:33.838697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:33.924613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a4443-0bda-4f9a-9248-acd59d1efca9 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:33.957313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75eae35-365e-4300-a271-29811b78c9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:33.957956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eae35-365e-4300-a271-29811b78c9fb None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:33.958241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eae35-365e-4300-a271-29811b78c9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:33.958519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eae35-365e-4300-a271-29811b78c9fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:34.023583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75eae35-365e-4300-a271-29811b78c9fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:34.024964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75eae35-365e-4300-a271-29811b78c9fb None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:34.038669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:34.121922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 574/1144] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:55:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:55:34.127852 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace25ecd-df79-41f4-aeef-a46ea2a132f4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:34.225915 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace25ecd-df79-41f4-aeef-a46ea2a132f4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:34.258487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4e671-552e-4978-8bd8-9e6b191f85f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:34.259330 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4e671-552e-4978-8bd8-9e6b191f85f5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:34.259636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4e671-552e-4978-8bd8-9e6b191f85f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:34.259910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4e671-552e-4978-8bd8-9e6b191f85f5 None admin] PATH_INFO: `/v3/projects/1d2f9d49a9ea453283967448d43efcc5` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:34.261180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4e671-552e-4978-8bd8-9e6b191f85f5 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2f9d49a9ea453283967448d43efcc5)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:34.262614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4e671-552e-4978-8bd8-9e6b191f85f5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:34.264325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 571/1145] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:55:34 2022] GET /identity/v3/projects/1d2f9d49a9ea453283967448d43efcc5 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:34.270933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75eae35-365e-4300-a271-29811b78c9fb None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:34.366600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75eae35-365e-4300-a271-29811b78c9fb None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:34.380337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:34.402423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4201b-46f1-4896-b118-863aa4738a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:34.403081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4201b-46f1-4896-b118-863aa4738a96 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:34.403363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4201b-46f1-4896-b118-863aa4738a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:34.403669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4201b-46f1-4896-b118-863aa4738a96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:34.404492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4201b-46f1-4896-b118-863aa4738a96 None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=1d2f9d49a9ea453283967448d43efcc5, user.id=69b2f1258aad4705a81520417ab6c864) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:34.404799 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4201b-46f1-4896-b118-863aa4738a96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:34.406226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4201b-46f1-4896-b118-863aa4738a96 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:34.411751 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 575/1146] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:55:34 2022] GET /identity/v3/role_assignments?user.id=69b2f1258aad4705a81520417ab6c864&scope.project.id=1d2f9d49a9ea453283967448d43efcc5&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:36.418538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a16dd-51e0-4d80-a858-d614b6b6aaf6 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:36.419083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a16dd-51e0-4d80-a858-d614b6b6aaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:36.419083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a16dd-51e0-4d80-a858-d614b6b6aaf6 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:36.420486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 572/1147] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:36.426119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71761c8-e09a-4754-aa9c-237191f6ae44 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:36.426578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71761c8-e09a-4754-aa9c-237191f6ae44 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:36.426679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71761c8-e09a-4754-aa9c-237191f6ae44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:36.444080 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c71761c8-e09a-4754-aa9c-237191f6ae44 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:36.445327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c71761c8-e09a-4754-aa9c-237191f6ae44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:36.510811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 576/1148] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:36.532849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a16dd-51e0-4d80-a858-d614b6b6aaf6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:36.548858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:36.635519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a16dd-51e0-4d80-a858-d614b6b6aaf6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:36.669494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8600da-1af3-494c-a5a3-538cb79b8a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:36.670282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8600da-1af3-494c-a5a3-538cb79b8a46 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:36.670568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8600da-1af3-494c-a5a3-538cb79b8a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:36.670847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8600da-1af3-494c-a5a3-538cb79b8a46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:36.681350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8600da-1af3-494c-a5a3-538cb79b8a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-4d8600da-1af3-494c-a5a3-538cb79b8a46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:55:36.689244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:36.690374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:36.691446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 573/1149] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:55:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:36.696598 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71761c8-e09a-4754-aa9c-237191f6ae44 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:36.793316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71761c8-e09a-4754-aa9c-237191f6ae44 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:36.801490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:36.831619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506f5dc-3d53-4091-a603-b9cac39ab5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:36.832281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506f5dc-3d53-4091-a603-b9cac39ab5fd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:36.832569 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506f5dc-3d53-4091-a603-b9cac39ab5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:36.832852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506f5dc-3d53-4091-a603-b9cac39ab5fd None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:36.833600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506f5dc-3d53-4091-a603-b9cac39ab5fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:36.833912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506f5dc-3d53-4091-a603-b9cac39ab5fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:36.835277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506f5dc-3d53-4091-a603-b9cac39ab5fd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:36.843143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 577/1150] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:55:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:36.849456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8600da-1af3-494c-a5a3-538cb79b8a46 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:36.921459 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:36.945394 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8600da-1af3-494c-a5a3-538cb79b8a46 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:36.978003 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f1040-4c40-450a-a28e-b048c0f30d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:36.978794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f1040-4c40-450a-a28e-b048c0f30d0d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:36.979076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f1040-4c40-450a-a28e-b048c0f30d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:36.979348 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f1040-4c40-450a-a28e-b048c0f30d0d None admin] PATH_INFO: `/v3/users/69b2f1258aad4705a81520417ab6c864` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:36.980541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f1040-4c40-450a-a28e-b048c0f30d0d None admin] RBAC: Authorizing `identity:get_user(user_id=69b2f1258aad4705a81520417ab6c864)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:36.981960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f1040-4c40-450a-a28e-b048c0f30d0d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:36.983575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 574/1151] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:36 2022] GET /identity/v3/users/69b2f1258aad4705a81520417ab6c864 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:36.989495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506f5dc-3d53-4091-a603-b9cac39ab5fd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:37.087495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506f5dc-3d53-4091-a603-b9cac39ab5fd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:37.120739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c909830-a5f5-4ccb-95de-4bfda4337f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:37.121581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c909830-a5f5-4ccb-95de-4bfda4337f66 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:37.121862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c909830-a5f5-4ccb-95de-4bfda4337f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:37.122135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c909830-a5f5-4ccb-95de-4bfda4337f66 None admin] PATH_INFO: `/v3/projects/1d2f9d49a9ea453283967448d43efcc5` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:37.123301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c909830-a5f5-4ccb-95de-4bfda4337f66 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2f9d49a9ea453283967448d43efcc5)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:37.124766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c909830-a5f5-4ccb-95de-4bfda4337f66 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:37.126448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 578/1152] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:55:36 2022] GET /identity/v3/projects/1d2f9d49a9ea453283967448d43efcc5 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:37.132412 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f1040-4c40-450a-a28e-b048c0f30d0d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:37.233798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f1040-4c40-450a-a28e-b048c0f30d0d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:37.265864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15661095-12c2-4685-badc-77d611763f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:37.266747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15661095-12c2-4685-badc-77d611763f49 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:37.267033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15661095-12c2-4685-badc-77d611763f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:37.267310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15661095-12c2-4685-badc-77d611763f49 None admin] PATH_INFO: `/v3/projects/1d2f9d49a9ea453283967448d43efcc5/users/69b2f1258aad4705a81520417ab6c864/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:37.269829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15661095-12c2-4685-badc-77d611763f49 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d2f9d49a9ea453283967448d43efcc5, user_id=69b2f1258aad4705a81520417ab6c864, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:37.271296 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15661095-12c2-4685-badc-77d611763f49 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:37.284203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 575/1153] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:55:37 2022] PUT /identity/v3/projects/1d2f9d49a9ea453283967448d43efcc5/users/69b2f1258aad4705a81520417ab6c864/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:55:39.278574 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e1337-5377-486f-9eb6-ff5c27110835 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:39.279109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e1337-5377-486f-9eb6-ff5c27110835 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:39.279109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e1337-5377-486f-9eb6-ff5c27110835 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:39.280507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 579/1154] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:39.285650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb656944-d8c7-4540-a4ce-54c937160299 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:39.286030 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb656944-d8c7-4540-a4ce-54c937160299 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:39.286218 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb656944-d8c7-4540-a4ce-54c937160299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:39.291209 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:39.309816 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bb656944-d8c7-4540-a4ce-54c937160299 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:39.311038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bb656944-d8c7-4540-a4ce-54c937160299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:39.378574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 576/1155] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:39.401843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e1337-5377-486f-9eb6-ff5c27110835 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:39.438292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:39.513593 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e1337-5377-486f-9eb6-ff5c27110835 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:39.547815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e67f72-265f-4fa6-a166-88ceabec6f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:39.548626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e67f72-265f-4fa6-a166-88ceabec6f1f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:39.548915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e67f72-265f-4fa6-a166-88ceabec6f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:39.549195 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e67f72-265f-4fa6-a166-88ceabec6f1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:39.560989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e67f72-265f-4fa6-a166-88ceabec6f1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c4e67f72-265f-4fa6-a166-88ceabec6f1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:55:39.568818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:39.569924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:39.570969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 580/1156] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:55:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:39.576160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb656944-d8c7-4540-a4ce-54c937160299 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:39.675997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb656944-d8c7-4540-a4ce-54c937160299 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:39.706937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:39.712457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdfff2-f166-4fbd-877a-b373606273ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:39.713136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdfff2-f166-4fbd-877a-b373606273ef None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:39.713419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdfff2-f166-4fbd-877a-b373606273ef None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:39.713697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdfff2-f166-4fbd-877a-b373606273ef None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:39.714455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffdfff2-f166-4fbd-877a-b373606273ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:39.714766 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffdfff2-f166-4fbd-877a-b373606273ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:39.716190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffdfff2-f166-4fbd-877a-b373606273ef None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:39.725766 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 577/1157] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:55:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:39.732491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e67f72-265f-4fa6-a166-88ceabec6f1f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:39.829793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:39.836760 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e67f72-265f-4fa6-a166-88ceabec6f1f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:39.870705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589eda87-bd47-422b-a906-5d78bd7b3d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:39.871600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589eda87-bd47-422b-a906-5d78bd7b3d48 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:39.871889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589eda87-bd47-422b-a906-5d78bd7b3d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:39.872171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589eda87-bd47-422b-a906-5d78bd7b3d48 None admin] PATH_INFO: `/v3/users/69b2f1258aad4705a81520417ab6c864` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:39.873346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589eda87-bd47-422b-a906-5d78bd7b3d48 None admin] RBAC: Authorizing `identity:get_user(user_id=69b2f1258aad4705a81520417ab6c864)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:39.874776 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589eda87-bd47-422b-a906-5d78bd7b3d48 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:39.876457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 581/1158] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:39 2022] GET /identity/v3/users/69b2f1258aad4705a81520417ab6c864 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:39.882185 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdfff2-f166-4fbd-877a-b373606273ef None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:39.978482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdfff2-f166-4fbd-877a-b373606273ef None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:40.009711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116ec3c-736c-4c1a-8a3f-0b9586ebdaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:40.010349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116ec3c-736c-4c1a-8a3f-0b9586ebdaf1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:40.010629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116ec3c-736c-4c1a-8a3f-0b9586ebdaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:40.010905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116ec3c-736c-4c1a-8a3f-0b9586ebdaf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:40.068424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:40.076009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116ec3c-736c-4c1a-8a3f-0b9586ebdaf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:40.077457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116ec3c-736c-4c1a-8a3f-0b9586ebdaf1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:40.172867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 578/1159] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:55:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:55:40.178719 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589eda87-bd47-422b-a906-5d78bd7b3d48 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:40.277048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589eda87-bd47-422b-a906-5d78bd7b3d48 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:40.309068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ad0be-23f9-4081-91be-063f44729362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:40.309919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ad0be-23f9-4081-91be-063f44729362 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:40.310207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ad0be-23f9-4081-91be-063f44729362 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:40.310488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ad0be-23f9-4081-91be-063f44729362 None admin] PATH_INFO: `/v3/projects/1d2f9d49a9ea453283967448d43efcc5` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:40.311679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ad0be-23f9-4081-91be-063f44729362 None admin] RBAC: Authorizing `identity:get_project(project_id=1d2f9d49a9ea453283967448d43efcc5)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:40.313118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ad0be-23f9-4081-91be-063f44729362 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:40.314806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 582/1160] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:55:40 2022] GET /identity/v3/projects/1d2f9d49a9ea453283967448d43efcc5 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:40.321295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116ec3c-736c-4c1a-8a3f-0b9586ebdaf1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:40.409977 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:40.416835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116ec3c-736c-4c1a-8a3f-0b9586ebdaf1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:40.449570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19892b3c-3a69-4034-8c5b-4b88b80c83f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:40.450146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19892b3c-3a69-4034-8c5b-4b88b80c83f9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:40.450431 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19892b3c-3a69-4034-8c5b-4b88b80c83f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:40.450709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19892b3c-3a69-4034-8c5b-4b88b80c83f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:40.451555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19892b3c-3a69-4034-8c5b-4b88b80c83f9 None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=1d2f9d49a9ea453283967448d43efcc5, user.id=69b2f1258aad4705a81520417ab6c864) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:40.451869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19892b3c-3a69-4034-8c5b-4b88b80c83f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:40.453300 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19892b3c-3a69-4034-8c5b-4b88b80c83f9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:40.459627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 579/1161] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:55:40 2022] GET /identity/v3/role_assignments?user.id=69b2f1258aad4705a81520417ab6c864&scope.project.id=1d2f9d49a9ea453283967448d43efcc5&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:42.516349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274b353-5cee-4a09-ada8-43119f136262 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:42.516868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274b353-5cee-4a09-ada8-43119f136262 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:42.516868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274b353-5cee-4a09-ada8-43119f136262 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:42.518778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 583/1162] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:42.523902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995d6c3-e3b8-4ab1-80d1-67aed967ff5e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:42.524273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995d6c3-e3b8-4ab1-80d1-67aed967ff5e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:42.524464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995d6c3-e3b8-4ab1-80d1-67aed967ff5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:42.541275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5995d6c3-e3b8-4ab1-80d1-67aed967ff5e None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:42.542458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5995d6c3-e3b8-4ab1-80d1-67aed967ff5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:42.607684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 580/1163] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:42.630229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274b353-5cee-4a09-ada8-43119f136262 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:42.661534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:42.731657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274b353-5cee-4a09-ada8-43119f136262 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:42.764395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059cef2-1ca9-42ec-8a08-1feaa3e31c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:42.765034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059cef2-1ca9-42ec-8a08-1feaa3e31c59 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:42.765317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059cef2-1ca9-42ec-8a08-1feaa3e31c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:42.765597 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059cef2-1ca9-42ec-8a08-1feaa3e31c59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:42.830008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059cef2-1ca9-42ec-8a08-1feaa3e31c59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:42.831972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059cef2-1ca9-42ec-8a08-1feaa3e31c59 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:42.860434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:42.928977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 584/1164] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:55:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:55:42.935206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995d6c3-e3b8-4ab1-80d1-67aed967ff5e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:43.018999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:43.031075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995d6c3-e3b8-4ab1-80d1-67aed967ff5e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:43.067208 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826483df-31a6-4bc8-89ba-4a7991ad4cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:43.068034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826483df-31a6-4bc8-89ba-4a7991ad4cfd None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:43.068322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826483df-31a6-4bc8-89ba-4a7991ad4cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:43.068601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826483df-31a6-4bc8-89ba-4a7991ad4cfd None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:43.075068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826483df-31a6-4bc8-89ba-4a7991ad4cfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:43.076550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826483df-31a6-4bc8-89ba-4a7991ad4cfd None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-826483df-31a6-4bc8-89ba-4a7991ad4cfd None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:43.090390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:43.091630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:43.092544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 581/1165] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Jul 14 10:55:42 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:43.097644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059cef2-1ca9-42ec-8a08-1feaa3e31c59 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:43.194927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059cef2-1ca9-42ec-8a08-1feaa3e31c59 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:43.223942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:43.229329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80ee73-8f05-4886-937e-2eee42b597f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:43.229992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80ee73-8f05-4886-937e-2eee42b597f9 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:43.230280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80ee73-8f05-4886-937e-2eee42b597f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:43.230562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80ee73-8f05-4886-937e-2eee42b597f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:43.231277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80ee73-8f05-4886-937e-2eee42b597f9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:43.231630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80ee73-8f05-4886-937e-2eee42b597f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:43.233011 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80ee73-8f05-4886-937e-2eee42b597f9 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:43.240753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 585/1166] 10.222.0.75 () {56 vars in 1115 bytes} [Thu Jul 14 10:55:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:45.231245 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61f7f5-6300-4a87-83b7-0fa6783d74b6 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:45.231791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61f7f5-6300-4a87-83b7-0fa6783d74b6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:45.231885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61f7f5-6300-4a87-83b7-0fa6783d74b6 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:45.233198 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 582/1167] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:45.238403 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc0e25-a31c-4022-a1c4-1e5ee15817cc None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:45.238837 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc0e25-a31c-4022-a1c4-1e5ee15817cc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:45.238967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc0e25-a31c-4022-a1c4-1e5ee15817cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:45.255204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a9fc0e25-a31c-4022-a1c4-1e5ee15817cc None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:45.256423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a9fc0e25-a31c-4022-a1c4-1e5ee15817cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:45.321553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 586/1168] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:55:45.344338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61f7f5-6300-4a87-83b7-0fa6783d74b6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:45.443795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61f7f5-6300-4a87-83b7-0fa6783d74b6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:45.476904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2693ab3-84b6-4313-a8f7-06eb3cf00781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:45.477496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2693ab3-84b6-4313-a8f7-06eb3cf00781 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:45.477784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2693ab3-84b6-4313-a8f7-06eb3cf00781 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:45.478066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2693ab3-84b6-4313-a8f7-06eb3cf00781 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:45.478806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2693ab3-84b6-4313-a8f7-06eb3cf00781 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:45.480215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2693ab3-84b6-4313-a8f7-06eb3cf00781 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:45.497849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 583/1169] 10.222.0.75 () {60 vars in 1138 bytes} [Thu Jul 14 10:55:45 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:55:47.555346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4cd9-41a6-4fc3-b52e-562b060e704d None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:47.555981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4cd9-41a6-4fc3-b52e-562b060e704d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:47.555981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4cd9-41a6-4fc3-b52e-562b060e704d None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:47.557373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 587/1170] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:47.562865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a3ac4-e7b4-4782-a518-82c738a85fa2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:47.563332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a3ac4-e7b4-4782-a518-82c738a85fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:47.563487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a3ac4-e7b4-4782-a518-82c738a85fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:47.580748 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8b6a3ac4-e7b4-4782-a518-82c738a85fa2 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:47.581948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8b6a3ac4-e7b4-4782-a518-82c738a85fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:47.583952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:47.651131 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 584/1171] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:47.675323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c4cd9-41a6-4fc3-b52e-562b060e704d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:47.778435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c4cd9-41a6-4fc3-b52e-562b060e704d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:47.804655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:47.813906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dc1d5-39d2-4ac3-9759-842b8f5df502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:47.814741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dc1d5-39d2-4ac3-9759-842b8f5df502 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:47.815068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dc1d5-39d2-4ac3-9759-842b8f5df502 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:47.815349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dc1d5-39d2-4ac3-9759-842b8f5df502 None admin] PATH_INFO: `/v3/domains/175784d0afef440ea1a30a214396497c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:47.826079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6dc1d5-39d2-4ac3-9759-842b8f5df502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=175784d0afef440ea1a30a214396497c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:47.827549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6dc1d5-39d2-4ac3-9759-842b8f5df502 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:47.829191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 588/1172] 10.222.0.75 () {56 vars in 1183 bytes} [Thu Jul 14 10:55:47 2022] GET /identity/v3/domains/175784d0afef440ea1a30a214396497c => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:55:47.835171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a3ac4-e7b4-4782-a518-82c738a85fa2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:47.933290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a3ac4-e7b4-4782-a518-82c738a85fa2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:47.967191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174089aa-0b50-4189-98dc-0dd7a2678c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:47.967877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174089aa-0b50-4189-98dc-0dd7a2678c6a None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:47.968171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174089aa-0b50-4189-98dc-0dd7a2678c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:47.968453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174089aa-0b50-4189-98dc-0dd7a2678c6a None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:47.969155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174089aa-0b50-4189-98dc-0dd7a2678c6a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:47.970593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174089aa-0b50-4189-98dc-0dd7a2678c6a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:47.978945 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:47.996232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 585/1173] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:55:47 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:55:50.095577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d2110-6b64-4f0d-9591-a07dbfb87e52 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:50.096137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d2110-6b64-4f0d-9591-a07dbfb87e52 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:50.096254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d2110-6b64-4f0d-9591-a07dbfb87e52 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:50.097550 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 589/1174] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:50.102744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843517c9-09d0-4ae3-b174-4888010ebe52 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:50.103158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843517c9-09d0-4ae3-b174-4888010ebe52 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:50.103312 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843517c9-09d0-4ae3-b174-4888010ebe52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:50.119909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-843517c9-09d0-4ae3-b174-4888010ebe52 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:50.121115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-843517c9-09d0-4ae3-b174-4888010ebe52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:50.187165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 586/1175] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:50.212231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d2110-6b64-4f0d-9591-a07dbfb87e52 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:50.311705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d2110-6b64-4f0d-9591-a07dbfb87e52 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:50.345154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc147b4c-8b26-4f72-a512-058c08fe89db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:50.346097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc147b4c-8b26-4f72-a512-058c08fe89db None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:50.346333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc147b4c-8b26-4f72-a512-058c08fe89db None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:50.346613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc147b4c-8b26-4f72-a512-058c08fe89db None admin] PATH_INFO: `/v3/domains/175784d0afef440ea1a30a214396497c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:50.347816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc147b4c-8b26-4f72-a512-058c08fe89db None admin] RBAC: Authorizing `identity:get_domain(domain_id=175784d0afef440ea1a30a214396497c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:50.349256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc147b4c-8b26-4f72-a512-058c08fe89db None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:50.350895 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 590/1176] 10.222.0.75 () {56 vars in 1183 bytes} [Thu Jul 14 10:55:50 2022] GET /identity/v3/domains/175784d0afef440ea1a30a214396497c => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:50.356787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843517c9-09d0-4ae3-b174-4888010ebe52 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:50.452866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843517c9-09d0-4ae3-b174-4888010ebe52 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:50.472214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:50.486352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c90b7c-aa6e-4eb7-a614-573b1057cd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:50.487584 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c90b7c-aa6e-4eb7-a614-573b1057cd06 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:50.487869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c90b7c-aa6e-4eb7-a614-573b1057cd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:50.488144 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c90b7c-aa6e-4eb7-a614-573b1057cd06 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:50.488855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c90b7c-aa6e-4eb7-a614-573b1057cd06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:50.490287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c90b7c-aa6e-4eb7-a614-573b1057cd06 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:50.511317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 587/1177] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:55:50 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:55:52.666599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b99237-0449-440d-b2e8-da0708da64c8 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:52.667154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b99237-0449-440d-b2e8-da0708da64c8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:52.667154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b99237-0449-440d-b2e8-da0708da64c8 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:52.668557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 591/1178] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:52.673702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54c788-5cb5-43de-b4b2-66be651e3577 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:52.674180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54c788-5cb5-43de-b4b2-66be651e3577 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:52.674285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54c788-5cb5-43de-b4b2-66be651e3577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:52.692592 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7e54c788-5cb5-43de-b4b2-66be651e3577 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:52.693814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7e54c788-5cb5-43de-b4b2-66be651e3577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:52.758820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 588/1179] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:52.782629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b99237-0449-440d-b2e8-da0708da64c8 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:52.841631 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:52.886666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b99237-0449-440d-b2e8-da0708da64c8 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:52.921333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e637060-286f-42ec-aff7-d9dbf44d1879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:52.922045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e637060-286f-42ec-aff7-d9dbf44d1879 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:52.922324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e637060-286f-42ec-aff7-d9dbf44d1879 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:52.922598 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e637060-286f-42ec-aff7-d9dbf44d1879 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:52.933340 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e637060-286f-42ec-aff7-d9dbf44d1879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-1e637060-286f-42ec-aff7-d9dbf44d1879 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:55:52.941423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:52.942614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:52.943937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 592/1180] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:55:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:52.948659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54c788-5cb5-43de-b4b2-66be651e3577 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:53.045675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54c788-5cb5-43de-b4b2-66be651e3577 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:53.053046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:53.081240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bf7a6-7bc1-48a8-87f9-92795e8bd480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:53.081908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bf7a6-7bc1-48a8-87f9-92795e8bd480 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:53.082195 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bf7a6-7bc1-48a8-87f9-92795e8bd480 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:53.082473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bf7a6-7bc1-48a8-87f9-92795e8bd480 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:53.083219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bf7a6-7bc1-48a8-87f9-92795e8bd480 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:53.083566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bf7a6-7bc1-48a8-87f9-92795e8bd480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:53.084939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bf7a6-7bc1-48a8-87f9-92795e8bd480 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:53.092935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 589/1181] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:55:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:53.098841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e637060-286f-42ec-aff7-d9dbf44d1879 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:53.197701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e637060-286f-42ec-aff7-d9dbf44d1879 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:53.211820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:53.232325 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26491d2-a233-4639-82ca-ed108953ed34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:53.233194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26491d2-a233-4639-82ca-ed108953ed34 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:53.233478 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26491d2-a233-4639-82ca-ed108953ed34 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:53.233756 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26491d2-a233-4639-82ca-ed108953ed34 None admin] PATH_INFO: `/v3/users/1cc3a64ac6a848e5928292fc885523d9` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:53.247722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26491d2-a233-4639-82ca-ed108953ed34 None admin] RBAC: Authorizing `identity:get_user(user_id=1cc3a64ac6a848e5928292fc885523d9)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:53.249192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26491d2-a233-4639-82ca-ed108953ed34 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:53.250893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 593/1182] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:53 2022] GET /identity/v3/users/1cc3a64ac6a848e5928292fc885523d9 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:53.256989 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bf7a6-7bc1-48a8-87f9-92795e8bd480 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:53.355980 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bf7a6-7bc1-48a8-87f9-92795e8bd480 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:53.387575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d84b7-6267-4b71-879e-24603ff750d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:53.388219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d84b7-6267-4b71-879e-24603ff750d0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:53.388507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d84b7-6267-4b71-879e-24603ff750d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:53.388783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d84b7-6267-4b71-879e-24603ff750d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:53.421882 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:53.453404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d84b7-6267-4b71-879e-24603ff750d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:53.454792 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d84b7-6267-4b71-879e-24603ff750d0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:53.550215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 590/1183] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:55:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:55:53.556102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26491d2-a233-4639-82ca-ed108953ed34 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:53.655944 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26491d2-a233-4639-82ca-ed108953ed34 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:53.690234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa42fa6-f1aa-452c-bf36-937862116ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:53.691086 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa42fa6-f1aa-452c-bf36-937862116ebd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:53.691371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa42fa6-f1aa-452c-bf36-937862116ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:53.691681 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa42fa6-f1aa-452c-bf36-937862116ebd None admin] PATH_INFO: `/v3/projects/af7ab6e130af465383d57f770c523f64` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:53.692848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa42fa6-f1aa-452c-bf36-937862116ebd None admin] RBAC: Authorizing `identity:get_project(project_id=af7ab6e130af465383d57f770c523f64)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:53.694269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa42fa6-f1aa-452c-bf36-937862116ebd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:53.695996 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 594/1184] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:55:53 2022] GET /identity/v3/projects/af7ab6e130af465383d57f770c523f64 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:53.703484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d84b7-6267-4b71-879e-24603ff750d0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:53.777140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:53.813832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d84b7-6267-4b71-879e-24603ff750d0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:53.857269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b348b7b-9514-4f63-a2a6-2a6f374dc6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:53.857968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348b7b-9514-4f63-a2a6-2a6f374dc6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:53.858256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348b7b-9514-4f63-a2a6-2a6f374dc6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:53.858573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348b7b-9514-4f63-a2a6-2a6f374dc6d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:53.859529 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b348b7b-9514-4f63-a2a6-2a6f374dc6d4 None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=af7ab6e130af465383d57f770c523f64, user.id=1cc3a64ac6a848e5928292fc885523d9) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:53.859890 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b348b7b-9514-4f63-a2a6-2a6f374dc6d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:53.861819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b348b7b-9514-4f63-a2a6-2a6f374dc6d4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:53.868645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 591/1185] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:55:53 2022] GET /identity/v3/role_assignments?user.id=1cc3a64ac6a848e5928292fc885523d9&scope.project.id=af7ab6e130af465383d57f770c523f64&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:55.884439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681ccde-cae4-40f4-901a-5f63b90c1215 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:55.884969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681ccde-cae4-40f4-901a-5f63b90c1215 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:55.885102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681ccde-cae4-40f4-901a-5f63b90c1215 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:55.886479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 595/1186] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:55.891692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8bfe9-fb1d-4c12-bd91-45d697bd1485 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:55.892133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8bfe9-fb1d-4c12-bd91-45d697bd1485 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:55.892252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8bfe9-fb1d-4c12-bd91-45d697bd1485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:55.910385 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ecd8bfe9-fb1d-4c12-bd91-45d697bd1485 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:55.911807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ecd8bfe9-fb1d-4c12-bd91-45d697bd1485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:55.979565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 592/1187] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:56.002664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681ccde-cae4-40f4-901a-5f63b90c1215 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:56.045291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:56.103473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681ccde-cae4-40f4-901a-5f63b90c1215 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:56.137347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32246452-21a2-48be-b2f7-4a852e08cfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:56.138079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32246452-21a2-48be-b2f7-4a852e08cfc7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:56.138375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32246452-21a2-48be-b2f7-4a852e08cfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:56.138659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32246452-21a2-48be-b2f7-4a852e08cfc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:56.150333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32246452-21a2-48be-b2f7-4a852e08cfc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-32246452-21a2-48be-b2f7-4a852e08cfc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:55:56.158312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:56.159542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:55:56.160537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 596/1188] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:55:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:56.165754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8bfe9-fb1d-4c12-bd91-45d697bd1485 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:56.226596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:56.263761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8bfe9-fb1d-4c12-bd91-45d697bd1485 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:56.295781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b2966-94b4-4d3c-aaf5-977d4d686cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:56.296443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b2966-94b4-4d3c-aaf5-977d4d686cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:56.296724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b2966-94b4-4d3c-aaf5-977d4d686cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:56.297000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b2966-94b4-4d3c-aaf5-977d4d686cc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:56.297731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b2966-94b4-4d3c-aaf5-977d4d686cc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:56.298034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b2966-94b4-4d3c-aaf5-977d4d686cc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:56.299450 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b2966-94b4-4d3c-aaf5-977d4d686cc7 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:56.307082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 593/1189] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:55:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:56.313082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32246452-21a2-48be-b2f7-4a852e08cfc7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:56.408527 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:56.411486 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32246452-21a2-48be-b2f7-4a852e08cfc7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:56.446763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d094c5-86e4-4566-8402-bdf5a7a7beb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:56.447669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d094c5-86e4-4566-8402-bdf5a7a7beb3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:56.447960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d094c5-86e4-4566-8402-bdf5a7a7beb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:56.448241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d094c5-86e4-4566-8402-bdf5a7a7beb3 None admin] PATH_INFO: `/v3/users/1cc3a64ac6a848e5928292fc885523d9` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:56.449428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d094c5-86e4-4566-8402-bdf5a7a7beb3 None admin] RBAC: Authorizing `identity:get_user(user_id=1cc3a64ac6a848e5928292fc885523d9)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:56.450861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d094c5-86e4-4566-8402-bdf5a7a7beb3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:56.452524 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 597/1190] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:56 2022] GET /identity/v3/users/1cc3a64ac6a848e5928292fc885523d9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:56.458497 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b2966-94b4-4d3c-aaf5-977d4d686cc7 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:56.558528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b2966-94b4-4d3c-aaf5-977d4d686cc7 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:56.581176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:56.591882 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ffb5f-b85f-420c-987a-88da782ab1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:56.592734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ffb5f-b85f-420c-987a-88da782ab1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:56.593017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ffb5f-b85f-420c-987a-88da782ab1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:56.593292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ffb5f-b85f-420c-987a-88da782ab1c5 None admin] PATH_INFO: `/v3/projects/af7ab6e130af465383d57f770c523f64` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:56.594460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ffb5f-b85f-420c-987a-88da782ab1c5 None admin] RBAC: Authorizing `identity:get_project(project_id=af7ab6e130af465383d57f770c523f64)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:56.595965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ffb5f-b85f-420c-987a-88da782ab1c5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:56.597648 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 594/1191] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:55:56 2022] GET /identity/v3/projects/af7ab6e130af465383d57f770c523f64 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:56.603596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d094c5-86e4-4566-8402-bdf5a7a7beb3 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:56.700852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d094c5-86e4-4566-8402-bdf5a7a7beb3 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:56.733189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5b0e8-ab61-445b-a10a-d3d3ba46bfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:56.734073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5b0e8-ab61-445b-a10a-d3d3ba46bfc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:56.734356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5b0e8-ab61-445b-a10a-d3d3ba46bfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:56.734633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5b0e8-ab61-445b-a10a-d3d3ba46bfc4 None admin] PATH_INFO: `/v3/projects/af7ab6e130af465383d57f770c523f64/users/1cc3a64ac6a848e5928292fc885523d9/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:56.737127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5b0e8-ab61-445b-a10a-d3d3ba46bfc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=af7ab6e130af465383d57f770c523f64, user_id=1cc3a64ac6a848e5928292fc885523d9, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:56.738554 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5b0e8-ab61-445b-a10a-d3d3ba46bfc4 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:56.750959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 598/1192] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:55:56 2022] PUT /identity/v3/projects/af7ab6e130af465383d57f770c523f64/users/1cc3a64ac6a848e5928292fc885523d9/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:55:58.754778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141161c5-301f-4506-a5e9-6f887ee617ed None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:58.755343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141161c5-301f-4506-a5e9-6f887ee617ed None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:58.755343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141161c5-301f-4506-a5e9-6f887ee617ed None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:58.756721 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 595/1193] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:55:58.761929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2259b-7604-45c3-b300-7fed511ee040 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:58.762370 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2259b-7604-45c3-b300-7fed511ee040 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:58.762479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2259b-7604-45c3-b300-7fed511ee040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:58.780076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-76f2259b-7604-45c3-b300-7fed511ee040 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:55:58.781271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-76f2259b-7604-45c3-b300-7fed511ee040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:55:58.805796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:58.852116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 599/1194] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:55:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:55:58.875749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141161c5-301f-4506-a5e9-6f887ee617ed None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:58.979625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141161c5-301f-4506-a5e9-6f887ee617ed None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:59.013318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9df5f-0a39-4590-9f7b-4d753b3f624a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:59.014028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9df5f-0a39-4590-9f7b-4d753b3f624a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:59.014321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9df5f-0a39-4590-9f7b-4d753b3f624a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:59.014600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9df5f-0a39-4590-9f7b-4d753b3f624a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:59.025263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc9df5f-0a39-4590-9f7b-4d753b3f624a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-0fc9df5f-0a39-4590-9f7b-4d753b3f624a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:55:59.032927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:55:59.034094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:55:59.035298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 596/1195] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:55:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:55:59.040188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2259b-7604-45c3-b300-7fed511ee040 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:59.137514 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2259b-7604-45c3-b300-7fed511ee040 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:59.171521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85e7bc-9614-4837-97f1-ca53f311956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:59.172195 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85e7bc-9614-4837-97f1-ca53f311956a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:59.174264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85e7bc-9614-4837-97f1-ca53f311956a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:59.174595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85e7bc-9614-4837-97f1-ca53f311956a None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:59.175356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85e7bc-9614-4837-97f1-ca53f311956a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:59.175695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85e7bc-9614-4837-97f1-ca53f311956a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:59.177079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85e7bc-9614-4837-97f1-ca53f311956a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:59.185852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 600/1196] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:55:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:59.192353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9df5f-0a39-4590-9f7b-4d753b3f624a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:59.245534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:59.293809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9df5f-0a39-4590-9f7b-4d753b3f624a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:59.327153 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f9166-5ed7-43be-b449-2ff78a3e403d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:59.328056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f9166-5ed7-43be-b449-2ff78a3e403d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:59.328343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f9166-5ed7-43be-b449-2ff78a3e403d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:59.328629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f9166-5ed7-43be-b449-2ff78a3e403d None admin] PATH_INFO: `/v3/users/1cc3a64ac6a848e5928292fc885523d9` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:59.329823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f9166-5ed7-43be-b449-2ff78a3e403d None admin] RBAC: Authorizing `identity:get_user(user_id=1cc3a64ac6a848e5928292fc885523d9)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:59.331881 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f9166-5ed7-43be-b449-2ff78a3e403d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:59.333516 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 597/1197] 10.222.0.75 () {56 vars in 1177 bytes} [Thu Jul 14 10:55:59 2022] GET /identity/v3/users/1cc3a64ac6a848e5928292fc885523d9 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:59.339367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85e7bc-9614-4837-97f1-ca53f311956a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:59.361823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:59.440109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85e7bc-9614-4837-97f1-ca53f311956a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:59.472922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6ffeb-b960-48ad-98dc-cf89f99b1b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:59.473575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6ffeb-b960-48ad-98dc-cf89f99b1b83 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:59.473859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6ffeb-b960-48ad-98dc-cf89f99b1b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:59.474136 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6ffeb-b960-48ad-98dc-cf89f99b1b83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:59.539917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6ffeb-b960-48ad-98dc-cf89f99b1b83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:59.541351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6ffeb-b960-48ad-98dc-cf89f99b1b83 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:59.562644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:59.639912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 601/1198] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:55:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:55:59.646365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f9166-5ed7-43be-b449-2ff78a3e403d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:59.762090 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f9166-5ed7-43be-b449-2ff78a3e403d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:59.786638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:59.802223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706e981-557b-4805-a00c-bd5b004e13a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:59.803159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706e981-557b-4805-a00c-bd5b004e13a5 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:59.803472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706e981-557b-4805-a00c-bd5b004e13a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:59.803753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706e981-557b-4805-a00c-bd5b004e13a5 None admin] PATH_INFO: `/v3/projects/af7ab6e130af465383d57f770c523f64` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:59.805018 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706e981-557b-4805-a00c-bd5b004e13a5 None admin] RBAC: Authorizing `identity:get_project(project_id=af7ab6e130af465383d57f770c523f64)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:59.806639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706e981-557b-4805-a00c-bd5b004e13a5 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:59.808714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 598/1199] 10.222.0.75 () {56 vars in 1186 bytes} [Thu Jul 14 10:55:59 2022] GET /identity/v3/projects/af7ab6e130af465383d57f770c523f64 => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:55:59.815896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6ffeb-b960-48ad-98dc-cf89f99b1b83 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:55:59.919364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6ffeb-b960-48ad-98dc-cf89f99b1b83 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:55:59.939026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:59.953562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73d469-b5cb-4d0e-a5f7-f469218ca259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:55:59.954221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73d469-b5cb-4d0e-a5f7-f469218ca259 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:55:59.954508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73d469-b5cb-4d0e-a5f7-f469218ca259 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:55:59.954787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73d469-b5cb-4d0e-a5f7-f469218ca259 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:55:59.955697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73d469-b5cb-4d0e-a5f7-f469218ca259 None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=af7ab6e130af465383d57f770c523f64, user.id=1cc3a64ac6a848e5928292fc885523d9) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:55:59.956014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73d469-b5cb-4d0e-a5f7-f469218ca259 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:55:59.957444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73d469-b5cb-4d0e-a5f7-f469218ca259 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:55:59.963405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 602/1200] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:55:59 2022] GET /identity/v3/role_assignments?user.id=1cc3a64ac6a848e5928292fc885523d9&scope.project.id=af7ab6e130af465383d57f770c523f64&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:09.798311 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=6 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:09.960584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=8 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:56.188635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9dd4c-7740-4605-a1c2-6ea593b33db9 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:56.189267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9dd4c-7740-4605-a1c2-6ea593b33db9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:56.189267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9dd4c-7740-4605-a1c2-6ea593b33db9 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:56.190623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 599/1201] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:56:56.198419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fff777-e628-46b3-8e67-0074e6d9f953 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:56.198419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fff777-e628-46b3-8e67-0074e6d9f953 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:56.198419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fff777-e628-46b3-8e67-0074e6d9f953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:56.221632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-83fff777-e628-46b3-8e67-0074e6d9f953 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:56.223109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-83fff777-e628-46b3-8e67-0074e6d9f953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:56.331441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 603/1202] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:56:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:56:56.351151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9dd4c-7740-4605-a1c2-6ea593b33db9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:56.500091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9dd4c-7740-4605-a1c2-6ea593b33db9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:56.546002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09127aa7-bc3b-4ceb-a0ad-d90b5c516190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:56.547179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09127aa7-bc3b-4ceb-a0ad-d90b5c516190 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:56.547532 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09127aa7-bc3b-4ceb-a0ad-d90b5c516190 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:56.547828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09127aa7-bc3b-4ceb-a0ad-d90b5c516190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:56.562471 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09127aa7-bc3b-4ceb-a0ad-d90b5c516190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:56.564641 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09127aa7-bc3b-4ceb-a0ad-d90b5c516190 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-09127aa7-bc3b-4ceb-a0ad-d90b5c516190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:56:56.587693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:56:56.588728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:56:56.589707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:56:56.589707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:56:56.589707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:56:56.589707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:56:56.589707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 600/1203] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:56:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:56:56.595748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fff777-e628-46b3-8e67-0074e6d9f953 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:56.725371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fff777-e628-46b3-8e67-0074e6d9f953 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:56.761753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:56.769445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86194690-792c-4afe-bee7-1faeb228c8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:56.770470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86194690-792c-4afe-bee7-1faeb228c8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:56.770758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86194690-792c-4afe-bee7-1faeb228c8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:56.771079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86194690-792c-4afe-bee7-1faeb228c8f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:56.771843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86194690-792c-4afe-bee7-1faeb228c8f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:56.772127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86194690-792c-4afe-bee7-1faeb228c8f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:56.773546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86194690-792c-4afe-bee7-1faeb228c8f2 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:56.788996 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 604/1204] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:56.793417 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09127aa7-bc3b-4ceb-a0ad-d90b5c516190 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:56.855744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:56.921929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09127aa7-bc3b-4ceb-a0ad-d90b5c516190 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:56.974617 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2596f9-d30b-4326-81c7-2e1754325f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:56.975979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2596f9-d30b-4326-81c7-2e1754325f51 None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:56.976393 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2596f9-d30b-4326-81c7-2e1754325f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:56.976799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2596f9-d30b-4326-81c7-2e1754325f51 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:56.977681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2596f9-d30b-4326-81c7-2e1754325f51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:56.979601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2596f9-d30b-4326-81c7-2e1754325f51 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:57.012846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 601/1205] 10.222.0.75 () {60 vars in 1132 bytes} [Thu Jul 14 10:56:56 2022] POST /identity/v3/users => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:59.197011 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61246385-7549-4c82-8e50-cc8b0ba4df8d None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:59.197011 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61246385-7549-4c82-8e50-cc8b0ba4df8d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:59.197442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61246385-7549-4c82-8e50-cc8b0ba4df8d None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:59.199846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 605/1206] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:56:59.205107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df10d7-f638-46eb-b17b-a06216ab0972 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:59.205598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df10d7-f638-46eb-b17b-a06216ab0972 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:59.205686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df10d7-f638-46eb-b17b-a06216ab0972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:59.223599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b1df10d7-f638-46eb-b17b-a06216ab0972 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:59.224820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b1df10d7-f638-46eb-b17b-a06216ab0972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:59.311696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 602/1207] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:56:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:56:59.337091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61246385-7549-4c82-8e50-cc8b0ba4df8d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:59.460636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61246385-7549-4c82-8e50-cc8b0ba4df8d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:59.511359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ebcb4-40ac-4ca9-9621-26f4bea2dbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:59.512567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ebcb4-40ac-4ca9-9621-26f4bea2dbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:59.512846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ebcb4-40ac-4ca9-9621-26f4bea2dbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:59.513146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ebcb4-40ac-4ca9-9621-26f4bea2dbd5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:59.526091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ebcb4-40ac-4ca9-9621-26f4bea2dbd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-166ebcb4-40ac-4ca9-9621-26f4bea2dbd5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:56:59.534564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:56:59.535665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:56:59.536534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 606/1208] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:56:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:56:59.542031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df10d7-f638-46eb-b17b-a06216ab0972 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:59.589155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:59.661080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df10d7-f638-46eb-b17b-a06216ab0972 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:59.702206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054624e-c18d-4089-9aaa-33d3fecc57e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:59.703124 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054624e-c18d-4089-9aaa-33d3fecc57e8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:59.703318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054624e-c18d-4089-9aaa-33d3fecc57e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:59.703778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054624e-c18d-4089-9aaa-33d3fecc57e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:59.705194 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054624e-c18d-4089-9aaa-33d3fecc57e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:59.705551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054624e-c18d-4089-9aaa-33d3fecc57e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:59.708601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054624e-c18d-4089-9aaa-33d3fecc57e8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:59.721450 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 603/1209] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:56:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:56:59.730438 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ebcb4-40ac-4ca9-9621-26f4bea2dbd5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:59.787870 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:59.852833 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ebcb4-40ac-4ca9-9621-26f4bea2dbd5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:59.913115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a372b78-95d3-4e41-b781-885e0716cd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:59.914278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a372b78-95d3-4e41-b781-885e0716cd9b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:59.914648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a372b78-95d3-4e41-b781-885e0716cd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:59.915087 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a372b78-95d3-4e41-b781-885e0716cd9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:59.930024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a372b78-95d3-4e41-b781-885e0716cd9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:59.931843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a372b78-95d3-4e41-b781-885e0716cd9b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-8a372b78-95d3-4e41-b781-885e0716cd9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:56:59.956006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:56:59.957016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:56:59.957912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:56:59.957912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:56:59.957912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:56:59.957912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:56:59.957912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 607/1210] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:56:59.963784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054624e-c18d-4089-9aaa-33d3fecc57e8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:00.112812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:00.115410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054624e-c18d-4089-9aaa-33d3fecc57e8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:00.158259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862603b3-3cb3-4c84-8357-99281816254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:00.159028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862603b3-3cb3-4c84-8357-99281816254c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:00.159434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862603b3-3cb3-4c84-8357-99281816254c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:00.159764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862603b3-3cb3-4c84-8357-99281816254c None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:00.160553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862603b3-3cb3-4c84-8357-99281816254c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:00.160898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862603b3-3cb3-4c84-8357-99281816254c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:00.162391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862603b3-3cb3-4c84-8357-99281816254c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:00.176533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 604/1211] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:00.182038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a372b78-95d3-4e41-b781-885e0716cd9b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:00.308558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a372b78-95d3-4e41-b781-885e0716cd9b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:00.311010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:00.356647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ec17e-3cd1-4596-b5b0-ba51f8bdb247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:00.357925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ec17e-3cd1-4596-b5b0-ba51f8bdb247 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:00.358227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ec17e-3cd1-4596-b5b0-ba51f8bdb247 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:00.358510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ec17e-3cd1-4596-b5b0-ba51f8bdb247 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:00.366624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ec17e-3cd1-4596-b5b0-ba51f8bdb247 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:00.368090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ec17e-3cd1-4596-b5b0-ba51f8bdb247 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-f39ec17e-3cd1-4596-b5b0-ba51f8bdb247 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:57:00.384333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:00.385699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:00.387200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:00.387200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:00.387200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:57:00.387200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:57:00.387200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:57:00.387200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:57:00.387200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:57:00.387200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:57:00.387200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:57:00.387200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:00.387200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 608/1212] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:00.391319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862603b3-3cb3-4c84-8357-99281816254c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:00.512565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862603b3-3cb3-4c84-8357-99281816254c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:00.551553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647b347-59f9-40e3-8a4f-97a17132a25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:00.552242 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647b347-59f9-40e3-8a4f-97a17132a25a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:00.552530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647b347-59f9-40e3-8a4f-97a17132a25a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:00.552810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647b347-59f9-40e3-8a4f-97a17132a25a None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:00.553653 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647b347-59f9-40e3-8a4f-97a17132a25a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:00.553963 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647b347-59f9-40e3-8a4f-97a17132a25a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:00.555458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647b347-59f9-40e3-8a4f-97a17132a25a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:00.575454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 605/1213] 10.222.0.75 () {56 vars in 1143 bytes} [Thu Jul 14 10:57:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:00.581915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ec17e-3cd1-4596-b5b0-ba51f8bdb247 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:00.706686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ec17e-3cd1-4596-b5b0-ba51f8bdb247 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:00.743255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71632e77-a352-4cf8-93fd-54c357a05793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:00.743932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71632e77-a352-4cf8-93fd-54c357a05793 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:00.744217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71632e77-a352-4cf8-93fd-54c357a05793 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:00.744495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71632e77-a352-4cf8-93fd-54c357a05793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:00.780533 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:00.827579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71632e77-a352-4cf8-93fd-54c357a05793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:00.829006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71632e77-a352-4cf8-93fd-54c357a05793 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:00.957895 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 609/1214] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:57:00.964478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647b347-59f9-40e3-8a4f-97a17132a25a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:00.983736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:01.079077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647b347-59f9-40e3-8a4f-97a17132a25a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:01.116399 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d1e87-b71f-430a-8dbe-a2e885d998be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:01.117301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d1e87-b71f-430a-8dbe-a2e885d998be None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:01.117587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d1e87-b71f-430a-8dbe-a2e885d998be None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:01.117868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d1e87-b71f-430a-8dbe-a2e885d998be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:01.129672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d1e87-b71f-430a-8dbe-a2e885d998be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:01.131152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d1e87-b71f-430a-8dbe-a2e885d998be None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-fb7d1e87-b71f-430a-8dbe-a2e885d998be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:01.154667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:01.155972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:01.157089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:01.157089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:01.157089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:01.157089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:01.159674 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 606/1215] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:01.162727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71632e77-a352-4cf8-93fd-54c357a05793 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:01.232023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:01.277863 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71632e77-a352-4cf8-93fd-54c357a05793 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:01.318433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab23aa-1fed-4ab9-ba85-cc863eb8f63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:01.319132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab23aa-1fed-4ab9-ba85-cc863eb8f63a None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:01.319419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab23aa-1fed-4ab9-ba85-cc863eb8f63a None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:01.319693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab23aa-1fed-4ab9-ba85-cc863eb8f63a None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:01.320419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab23aa-1fed-4ab9-ba85-cc863eb8f63a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:01.320733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab23aa-1fed-4ab9-ba85-cc863eb8f63a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:01.322119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab23aa-1fed-4ab9-ba85-cc863eb8f63a None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:01.337443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 610/1216] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:01.343674 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d1e87-b71f-430a-8dbe-a2e885d998be None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:01.429966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:01.465253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d1e87-b71f-430a-8dbe-a2e885d998be None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:01.503163 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9c895-2698-4d5a-afd7-f740643b3a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:01.503928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9c895-2698-4d5a-afd7-f740643b3a46 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:01.504210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9c895-2698-4d5a-afd7-f740643b3a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:01.504483 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9c895-2698-4d5a-afd7-f740643b3a46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:01.513389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9c895-2698-4d5a-afd7-f740643b3a46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:01.514927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9c895-2698-4d5a-afd7-f740643b3a46 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-c0a9c895-2698-4d5a-afd7-f740643b3a46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:57:01.531657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:01.534180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:01.535159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:57:01.535159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:57:01.535159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:57:01.535159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:57:01.535159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:57:01.535159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:57:01.535159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:57:01.535159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:01.535159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 607/1217] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:57:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:01.539586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab23aa-1fed-4ab9-ba85-cc863eb8f63a None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:01.674131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab23aa-1fed-4ab9-ba85-cc863eb8f63a None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:01.699099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:01.713147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d41391-0847-4cd9-b9fc-744b849b88b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:01.714026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d41391-0847-4cd9-b9fc-744b849b88b7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:01.714387 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d41391-0847-4cd9-b9fc-744b849b88b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:01.714761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d41391-0847-4cd9-b9fc-744b849b88b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:01.715804 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d41391-0847-4cd9-b9fc-744b849b88b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:01.716183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d41391-0847-4cd9-b9fc-744b849b88b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:01.717816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d41391-0847-4cd9-b9fc-744b849b88b7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:01.732625 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 611/1218] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:57:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:01.738523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9c895-2698-4d5a-afd7-f740643b3a46 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:01.859930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9c895-2698-4d5a-afd7-f740643b3a46 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:01.903874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:01.913658 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9213c2-ce01-41d4-956e-c35d2c0428d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:01.914356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9213c2-ce01-41d4-956e-c35d2c0428d8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:01.914656 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9213c2-ce01-41d4-956e-c35d2c0428d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:01.915021 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9213c2-ce01-41d4-956e-c35d2c0428d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:01.916043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9213c2-ce01-41d4-956e-c35d2c0428d8 None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=0fc1f2abce4845219741f4f48bf32d36) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:01.916348 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9213c2-ce01-41d4-956e-c35d2c0428d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:01.918010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9213c2-ce01-41d4-956e-c35d2c0428d8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:01.925045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 608/1219] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:01 2022] GET /identity/v3/role_assignments?user.id=0fc1f2abce4845219741f4f48bf32d36&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:04.106924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2827e-e082-4ce2-b540-6bddc9936cd9 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:04.107508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2827e-e082-4ce2-b540-6bddc9936cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:04.107508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2827e-e082-4ce2-b540-6bddc9936cd9 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:04.109049 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 612/1220] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:04.114809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3844d9b-82b7-455f-a367-e90ee49feaa3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:04.115292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3844d9b-82b7-455f-a367-e90ee49feaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:04.115416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3844d9b-82b7-455f-a367-e90ee49feaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:04.136220 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b3844d9b-82b7-455f-a367-e90ee49feaa3 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:04.137540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b3844d9b-82b7-455f-a367-e90ee49feaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:04.229150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 609/1221] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:57:04.253727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2827e-e082-4ce2-b540-6bddc9936cd9 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:04.371329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2827e-e082-4ce2-b540-6bddc9936cd9 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:04.408192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd9b04-6cbb-4773-89be-d1e75dc6f3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:04.409027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd9b04-6cbb-4773-89be-d1e75dc6f3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:04.409309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd9b04-6cbb-4773-89be-d1e75dc6f3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:04.409580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd9b04-6cbb-4773-89be-d1e75dc6f3d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:04.420753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd9b04-6cbb-4773-89be-d1e75dc6f3d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-6edd9b04-6cbb-4773-89be-d1e75dc6f3d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:04.429204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:04.430344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:04.431234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 613/1222] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:57:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:04.436832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3844d9b-82b7-455f-a367-e90ee49feaa3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:04.540211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3844d9b-82b7-455f-a367-e90ee49feaa3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:04.566678 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:04.580825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a794a7c-1588-4486-8aec-8f0666f7939c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:04.581515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a794a7c-1588-4486-8aec-8f0666f7939c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:04.581805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a794a7c-1588-4486-8aec-8f0666f7939c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:04.582086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a794a7c-1588-4486-8aec-8f0666f7939c None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:04.582849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a794a7c-1588-4486-8aec-8f0666f7939c None admin] RBAC: Adding query filter params (name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:04.583160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a794a7c-1588-4486-8aec-8f0666f7939c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:04.584585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a794a7c-1588-4486-8aec-8f0666f7939c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:04.593626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 610/1223] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:57:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:04.601840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd9b04-6cbb-4773-89be-d1e75dc6f3d5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:04.652017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:04.717663 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd9b04-6cbb-4773-89be-d1e75dc6f3d5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:04.758178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f3049-1068-41ee-bcb5-2aef1bcd6072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:04.758984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f3049-1068-41ee-bcb5-2aef1bcd6072 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:04.759267 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f3049-1068-41ee-bcb5-2aef1bcd6072 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:04.759577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f3049-1068-41ee-bcb5-2aef1bcd6072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:04.773479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f3049-1068-41ee-bcb5-2aef1bcd6072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:04.775237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f3049-1068-41ee-bcb5-2aef1bcd6072 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-466f3049-1068-41ee-bcb5-2aef1bcd6072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:04.796542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:04.798056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:04.799251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:04.799251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:04.799251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:04.799251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:04.799251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 614/1224] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:04.804925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a794a7c-1588-4486-8aec-8f0666f7939c None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:04.928797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a794a7c-1588-4486-8aec-8f0666f7939c None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:04.966661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b607bc-1649-457c-ab63-bb3334ddc07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:04.967334 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b607bc-1649-457c-ab63-bb3334ddc07e None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:04.967646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b607bc-1649-457c-ab63-bb3334ddc07e None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:04.967928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b607bc-1649-457c-ab63-bb3334ddc07e None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:04.968654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b607bc-1649-457c-ab63-bb3334ddc07e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:04.968960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b607bc-1649-457c-ab63-bb3334ddc07e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:04.970345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b607bc-1649-457c-ab63-bb3334ddc07e None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:04.985202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 611/1225] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:04.991506 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f3049-1068-41ee-bcb5-2aef1bcd6072 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:05.103846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:05.112473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f3049-1068-41ee-bcb5-2aef1bcd6072 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:05.150966 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fc6ae-2f79-4ee3-ab1e-131a39813d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:05.151817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fc6ae-2f79-4ee3-ab1e-131a39813d21 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:05.152106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fc6ae-2f79-4ee3-ab1e-131a39813d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:05.152389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4fc6ae-2f79-4ee3-ab1e-131a39813d21 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:05.160366 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4fc6ae-2f79-4ee3-ab1e-131a39813d21 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:05.161814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4fc6ae-2f79-4ee3-ab1e-131a39813d21 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-fc4fc6ae-2f79-4ee3-ab1e-131a39813d21 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:57:05.176591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:05.177772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:05.178752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:05.178752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:05.178752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:57:05.178752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:57:05.178752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:57:05.178752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:57:05.178752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:57:05.178752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:57:05.178752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:57:05.178752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:05.178752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 615/1226] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:05.184221 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b607bc-1649-457c-ab63-bb3334ddc07e None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:05.237644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:05.296244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b607bc-1649-457c-ab63-bb3334ddc07e None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:05.332624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f1bac-8974-4bcc-8429-e4bf7ec47cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:05.333307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f1bac-8974-4bcc-8429-e4bf7ec47cbe None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:05.333591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f1bac-8974-4bcc-8429-e4bf7ec47cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:05.333870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f1bac-8974-4bcc-8429-e4bf7ec47cbe None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:05.334711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f1bac-8974-4bcc-8429-e4bf7ec47cbe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:05.335026 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f1bac-8974-4bcc-8429-e4bf7ec47cbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:05.336502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f1bac-8974-4bcc-8429-e4bf7ec47cbe None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:05.353869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 612/1227] 10.222.0.75 () {56 vars in 1143 bytes} [Thu Jul 14 10:57:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:05.361407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fc6ae-2f79-4ee3-ab1e-131a39813d21 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:05.471613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4fc6ae-2f79-4ee3-ab1e-131a39813d21 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:05.508752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd639ad-00c1-456c-a297-f431f71907b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:05.509689 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd639ad-00c1-456c-a297-f431f71907b5 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:05.510014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd639ad-00c1-456c-a297-f431f71907b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:05.510258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd639ad-00c1-456c-a297-f431f71907b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:05.521324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd639ad-00c1-456c-a297-f431f71907b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:05.522771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd639ad-00c1-456c-a297-f431f71907b5 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-bfd639ad-00c1-456c-a297-f431f71907b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:05.539795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:05.540992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:05.547053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:05.547053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:05.547053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:05.547053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:05.547053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 616/1228] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:05.547469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f1bac-8974-4bcc-8429-e4bf7ec47cbe None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:05.654964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f1bac-8974-4bcc-8429-e4bf7ec47cbe None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:05.658052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:05.691922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b66e7-9070-439c-bdde-08424a3afa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:05.692586 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b66e7-9070-439c-bdde-08424a3afa8d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:05.692868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b66e7-9070-439c-bdde-08424a3afa8d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:05.693144 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b66e7-9070-439c-bdde-08424a3afa8d None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:05.693858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b66e7-9070-439c-bdde-08424a3afa8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:05.694168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b66e7-9070-439c-bdde-08424a3afa8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:05.695563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b66e7-9070-439c-bdde-08424a3afa8d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:05.707016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 613/1229] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:05.712811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd639ad-00c1-456c-a297-f431f71907b5 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:05.723348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:05.823818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd639ad-00c1-456c-a297-f431f71907b5 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:05.862521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f123654-5d7e-4900-a2ca-5a62695af86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:05.863374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f123654-5d7e-4900-a2ca-5a62695af86d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:05.863725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f123654-5d7e-4900-a2ca-5a62695af86d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:05.864076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f123654-5d7e-4900-a2ca-5a62695af86d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:05.872302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f123654-5d7e-4900-a2ca-5a62695af86d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:05.873911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f123654-5d7e-4900-a2ca-5a62695af86d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-1f123654-5d7e-4900-a2ca-5a62695af86d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:57:05.889872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:05.896567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:05.897758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:57:05.897758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:57:05.897758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:57:05.897758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:57:05.897758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:57:05.897758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:57:05.897758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:57:05.897758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:05.897758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 617/1230] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:57:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:05.899343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b66e7-9070-439c-bdde-08424a3afa8d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:06.012323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b66e7-9070-439c-bdde-08424a3afa8d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:06.049731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186c201-a713-41e7-a486-fc7dfaa38757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:06.050690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186c201-a713-41e7-a486-fc7dfaa38757 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:06.051002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186c201-a713-41e7-a486-fc7dfaa38757 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:06.051486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186c201-a713-41e7-a486-fc7dfaa38757 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:06.054287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186c201-a713-41e7-a486-fc7dfaa38757 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:06.054287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186c201-a713-41e7-a486-fc7dfaa38757 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:06.054621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186c201-a713-41e7-a486-fc7dfaa38757 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:06.070140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 614/1231] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:57:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:06.076766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f123654-5d7e-4900-a2ca-5a62695af86d None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:06.153938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:06.196114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f123654-5d7e-4900-a2ca-5a62695af86d None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:06.241084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3209019-cdb2-4bf3-866f-61e8377e9343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:06.242019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3209019-cdb2-4bf3-866f-61e8377e9343 None admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:06.242349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3209019-cdb2-4bf3-866f-61e8377e9343 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:06.242734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3209019-cdb2-4bf3-866f-61e8377e9343 None admin] PATH_INFO: `/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/0fc1f2abce4845219741f4f48bf32d36/roles/be88bd87b07244debee4e9545284ff13` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:06.259709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3209019-cdb2-4bf3-866f-61e8377e9343 None admin] RBAC: Authorizing `identity:create_grant(project_id=a68601f16d1c49548b6e3af0b9819ba3, user_id=0fc1f2abce4845219741f4f48bf32d36, role_id=be88bd87b07244debee4e9545284ff13)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:06.261184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3209019-cdb2-4bf3-866f-61e8377e9343 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:06.275070 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 618/1232] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:57:06 2022] PUT /identity/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/0fc1f2abce4845219741f4f48bf32d36/roles/be88bd87b07244debee4e9545284ff13 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:08.389694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5caff-e955-4767-bff1-057d98e57065 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:08.390308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5caff-e955-4767-bff1-057d98e57065 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:08.390308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5caff-e955-4767-bff1-057d98e57065 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:08.392184 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 615/1233] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:57:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:08.397599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b76a71-43e8-44b8-9639-95700573cd7c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:08.398129 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b76a71-43e8-44b8-9639-95700573cd7c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:08.398271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b76a71-43e8-44b8-9639-95700573cd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:08.416452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a4b76a71-43e8-44b8-9639-95700573cd7c None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:08.417724 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a4b76a71-43e8-44b8-9639-95700573cd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:08.488590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 619/1234] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:57:08.511853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5caff-e955-4767-bff1-057d98e57065 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:08.539629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:08.622801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5caff-e955-4767-bff1-057d98e57065 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:08.662467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02806750-8417-40c1-8395-9e175c783e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:08.663293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02806750-8417-40c1-8395-9e175c783e46 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:08.663605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02806750-8417-40c1-8395-9e175c783e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:08.664097 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02806750-8417-40c1-8395-9e175c783e46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:08.678963 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02806750-8417-40c1-8395-9e175c783e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-02806750-8417-40c1-8395-9e175c783e46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:08.690692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:08.692193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:08.693162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:08.693162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:08.693162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:08.693162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:08.693162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:08.693162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:08.693162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:08.693162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:08.693162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:08.693162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:08.693162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:57:08.693162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:08.699162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 616/1235] 10.222.0.75 () {56 vars in 1102 bytes} [Thu Jul 14 10:57:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:08.700402 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b76a71-43e8-44b8-9639-95700573cd7c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:08.761829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:08.833972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b76a71-43e8-44b8-9639-95700573cd7c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:08.877330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814a851-8a26-443e-a2da-f772520a0373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:08.878131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814a851-8a26-443e-a2da-f772520a0373 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:08.878493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814a851-8a26-443e-a2da-f772520a0373 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:08.878821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814a851-8a26-443e-a2da-f772520a0373 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:08.879731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814a851-8a26-443e-a2da-f772520a0373 None admin] RBAC: Adding query filter params (name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:08.880095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814a851-8a26-443e-a2da-f772520a0373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:08.882048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814a851-8a26-443e-a2da-f772520a0373 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:08.892222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 620/1236] 10.222.0.75 () {56 vars in 1103 bytes} [Thu Jul 14 10:57:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:08.897743 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02806750-8417-40c1-8395-9e175c783e46 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:08.992487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:09.009521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02806750-8417-40c1-8395-9e175c783e46 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:09.044056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ce18a-3752-4c32-8f3c-1787824c1220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:09.044761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ce18a-3752-4c32-8f3c-1787824c1220 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:09.045042 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ce18a-3752-4c32-8f3c-1787824c1220 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:09.045314 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ce18a-3752-4c32-8f3c-1787824c1220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:09.056823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ce18a-3752-4c32-8f3c-1787824c1220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:09.058730 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ce18a-3752-4c32-8f3c-1787824c1220 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-6e0ce18a-3752-4c32-8f3c-1787824c1220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:09.076662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:09.077782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:09.078719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:09.078719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:09.078719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:09.078719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:09.078719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 617/1237] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:09.084420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814a851-8a26-443e-a2da-f772520a0373 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:09.186733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814a851-8a26-443e-a2da-f772520a0373 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:09.197173 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:09.227314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59191ed0-5d55-40c8-9238-6e1f3e7694f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:09.227987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59191ed0-5d55-40c8-9238-6e1f3e7694f0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:09.228277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59191ed0-5d55-40c8-9238-6e1f3e7694f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:09.228581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59191ed0-5d55-40c8-9238-6e1f3e7694f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:09.229290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59191ed0-5d55-40c8-9238-6e1f3e7694f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:09.229604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59191ed0-5d55-40c8-9238-6e1f3e7694f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:09.230993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59191ed0-5d55-40c8-9238-6e1f3e7694f0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:09.243337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 621/1238] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:09.248900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ce18a-3752-4c32-8f3c-1787824c1220 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:09.354111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ce18a-3752-4c32-8f3c-1787824c1220 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:09.399689 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8c57c-9e3e-4038-a326-5153c54db2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:09.399689 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8c57c-9e3e-4038-a326-5153c54db2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:09.400427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8c57c-9e3e-4038-a326-5153c54db2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:09.400427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8c57c-9e3e-4038-a326-5153c54db2a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:09.404189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:09.409818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8c57c-9e3e-4038-a326-5153c54db2a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:09.411523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8c57c-9e3e-4038-a326-5153c54db2a8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-8cc8c57c-9e3e-4038-a326-5153c54db2a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:57:09.428287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:09.429524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:09.430711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:09.430711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:09.430711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:57:09.430711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:57:09.430711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:57:09.430711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:57:09.430711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:57:09.430711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:57:09.430711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:57:09.430711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:09.431735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 618/1239] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:09.437745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59191ed0-5d55-40c8-9238-6e1f3e7694f0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:09.550581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59191ed0-5d55-40c8-9238-6e1f3e7694f0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:09.591040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fba49-e4f9-4eb2-a72a-d1a47b686486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:09.591762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fba49-e4f9-4eb2-a72a-d1a47b686486 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:09.592181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fba49-e4f9-4eb2-a72a-d1a47b686486 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:09.592461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fba49-e4f9-4eb2-a72a-d1a47b686486 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:09.593582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81fba49-e4f9-4eb2-a72a-d1a47b686486 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:09.593900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81fba49-e4f9-4eb2-a72a-d1a47b686486 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:09.595710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81fba49-e4f9-4eb2-a72a-d1a47b686486 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:09.618989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 622/1240] 10.222.0.75 () {56 vars in 1143 bytes} [Thu Jul 14 10:57:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:09.624773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8c57c-9e3e-4038-a326-5153c54db2a8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:09.750694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8c57c-9e3e-4038-a326-5153c54db2a8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:09.788829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc64ca-1ba1-4fe9-b8fd-72a419e10550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:09.789737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc64ca-1ba1-4fe9-b8fd-72a419e10550 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:09.790058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc64ca-1ba1-4fe9-b8fd-72a419e10550 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:09.790405 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc64ca-1ba1-4fe9-b8fd-72a419e10550 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:09.862300 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:09.870293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc64ca-1ba1-4fe9-b8fd-72a419e10550 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:09.871836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc64ca-1ba1-4fe9-b8fd-72a419e10550 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:09.988044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 619/1241] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:57:09.994892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fba49-e4f9-4eb2-a72a-d1a47b686486 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:10.026700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:10.109334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fba49-e4f9-4eb2-a72a-d1a47b686486 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:10.149196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e583ff-0357-462f-93c2-5d2a2c065f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:10.150123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e583ff-0357-462f-93c2-5d2a2c065f33 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:10.150410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e583ff-0357-462f-93c2-5d2a2c065f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:10.150691 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e583ff-0357-462f-93c2-5d2a2c065f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:10.163661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e583ff-0357-462f-93c2-5d2a2c065f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:10.165140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e583ff-0357-462f-93c2-5d2a2c065f33 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-f0e583ff-0357-462f-93c2-5d2a2c065f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:10.185127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:10.186210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:10.187139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:10.187139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:10.187139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:10.187139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:10.187139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 623/1242] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:10.192567 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc64ca-1ba1-4fe9-b8fd-72a419e10550 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:10.302556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:10.310143 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc64ca-1ba1-4fe9-b8fd-72a419e10550 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:10.347706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b69792-0856-45cd-8e99-5a827ad964e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:10.348392 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b69792-0856-45cd-8e99-5a827ad964e8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:10.348671 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b69792-0856-45cd-8e99-5a827ad964e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:10.348945 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b69792-0856-45cd-8e99-5a827ad964e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:10.349680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b69792-0856-45cd-8e99-5a827ad964e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:10.349985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b69792-0856-45cd-8e99-5a827ad964e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:10.351410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b69792-0856-45cd-8e99-5a827ad964e8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:10.365641 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 620/1243] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:10.371344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e583ff-0357-462f-93c2-5d2a2c065f33 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:10.465472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:10.492023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e583ff-0357-462f-93c2-5d2a2c065f33 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:10.533129 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8ce7c-5096-466d-bce4-a0c2e39e2989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:10.533946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8ce7c-5096-466d-bce4-a0c2e39e2989 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:10.534228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8ce7c-5096-466d-bce4-a0c2e39e2989 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:10.534501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8ce7c-5096-466d-bce4-a0c2e39e2989 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:10.541989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8ce7c-5096-466d-bce4-a0c2e39e2989 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:10.543488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8ce7c-5096-466d-bce4-a0c2e39e2989 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-c7e8ce7c-5096-466d-bce4-a0c2e39e2989 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:57:10.559563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:10.560866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:10.562516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:57:10.562516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:57:10.562516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:57:10.562516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:57:10.562516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:57:10.562516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:57:10.562516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:57:10.562516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:10.563953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 624/1244] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:57:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:10.566992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b69792-0856-45cd-8e99-5a827ad964e8 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:10.684494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b69792-0856-45cd-8e99-5a827ad964e8 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:10.722265 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6349f-3de3-4b46-8398-094446645726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:10.722859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6349f-3de3-4b46-8398-094446645726 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:10.723162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6349f-3de3-4b46-8398-094446645726 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:10.723463 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6349f-3de3-4b46-8398-094446645726 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:10.724259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6349f-3de3-4b46-8398-094446645726 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:10.724572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6349f-3de3-4b46-8398-094446645726 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:10.726048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6349f-3de3-4b46-8398-094446645726 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:10.738834 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 621/1245] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:57:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:10.745317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8ce7c-5096-466d-bce4-a0c2e39e2989 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:10.864122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8ce7c-5096-466d-bce4-a0c2e39e2989 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:10.904666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f0f16-e79b-45de-89bc-06323bf0bce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:10.905490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f0f16-e79b-45de-89bc-06323bf0bce3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:10.905807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f0f16-e79b-45de-89bc-06323bf0bce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:10.906097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f0f16-e79b-45de-89bc-06323bf0bce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:10.907937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f0f16-e79b-45de-89bc-06323bf0bce3 None admin] RBAC: Adding query filter params (role.id=be88bd87b07244debee4e9545284ff13, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=0fc1f2abce4845219741f4f48bf32d36) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:10.908485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f0f16-e79b-45de-89bc-06323bf0bce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:10.910368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f0f16-e79b-45de-89bc-06323bf0bce3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:10.919310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 625/1246] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:10 2022] GET /identity/v3/role_assignments?user.id=0fc1f2abce4845219741f4f48bf32d36&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=be88bd87b07244debee4e9545284ff13 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:13.211376 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bba8e-32bd-4291-a665-41d549261b11 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.211947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bba8e-32bd-4291-a665-41d549261b11 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.211947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bba8e-32bd-4291-a665-41d549261b11 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.213291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 622/1247] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:13.218606 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fb1fa-da6e-4d1a-ab19-63b4aa0ce24a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.219089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fb1fa-da6e-4d1a-ab19-63b4aa0ce24a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.219210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fb1fa-da6e-4d1a-ab19-63b4aa0ce24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.225013 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:13.239592 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f81fb1fa-da6e-4d1a-ab19-63b4aa0ce24a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:13.240941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f81fb1fa-da6e-4d1a-ab19-63b4aa0ce24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:13.320856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 626/1248] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:57:13.346325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bba8e-32bd-4291-a665-41d549261b11 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.373197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:13.495154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bba8e-32bd-4291-a665-41d549261b11 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.535645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4c196-4470-4d5a-ac66-3c1bd36e71e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.536566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4c196-4470-4d5a-ac66-3c1bd36e71e1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.536903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4c196-4470-4d5a-ac66-3c1bd36e71e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.537214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4c196-4470-4d5a-ac66-3c1bd36e71e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.550405 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4c196-4470-4d5a-ac66-3c1bd36e71e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-72b4c196-4470-4d5a-ac66-3c1bd36e71e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:13.558578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:13.559847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:13.560907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:13.560907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:13.560907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:13.560907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:13.560907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:13.560907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:13.560907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:13.560907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:13.560907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:13.560907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:13.560907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:13.560907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:13.561714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 623/1249] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:57:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:13.567949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fb1fa-da6e-4d1a-ab19-63b4aa0ce24a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.689818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fb1fa-da6e-4d1a-ab19-63b4aa0ce24a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.726166 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:13.732340 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb298e3-d21c-45f6-83da-234b357f1bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.733296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb298e3-d21c-45f6-83da-234b357f1bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.733682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb298e3-d21c-45f6-83da-234b357f1bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.734173 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb298e3-d21c-45f6-83da-234b357f1bf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.735057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb298e3-d21c-45f6-83da-234b357f1bf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:13.735448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb298e3-d21c-45f6-83da-234b357f1bf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.737018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb298e3-d21c-45f6-83da-234b357f1bf0 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.745884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 627/1250] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:13.751836 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4c196-4470-4d5a-ac66-3c1bd36e71e1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.848201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:13.875571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4c196-4470-4d5a-ac66-3c1bd36e71e1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.915671 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7d6af-08bf-465f-8de6-084001b7b533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.916479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7d6af-08bf-465f-8de6-084001b7b533 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.916845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7d6af-08bf-465f-8de6-084001b7b533 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.917101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7d6af-08bf-465f-8de6-084001b7b533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.929063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7d6af-08bf-465f-8de6-084001b7b533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.930637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7d6af-08bf-465f-8de6-084001b7b533 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-a2c7d6af-08bf-465f-8de6-084001b7b533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:13.951197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:13.952371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:13.953319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:13.953319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:13.953319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:13.953319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:13.953319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 624/1251] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:13.959244 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb298e3-d21c-45f6-83da-234b357f1bf0 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.082036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb298e3-d21c-45f6-83da-234b357f1bf0 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.124514 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982734b-e079-4ee6-b07f-31158bf7e71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.125204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982734b-e079-4ee6-b07f-31158bf7e71b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.125492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982734b-e079-4ee6-b07f-31158bf7e71b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.125772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982734b-e079-4ee6-b07f-31158bf7e71b None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.127093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982734b-e079-4ee6-b07f-31158bf7e71b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:14.127443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982734b-e079-4ee6-b07f-31158bf7e71b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.128886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982734b-e079-4ee6-b07f-31158bf7e71b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.143032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 628/1252] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:14.149655 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7d6af-08bf-465f-8de6-084001b7b533 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.266252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7d6af-08bf-465f-8de6-084001b7b533 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.299866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:14.310457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19e2e2-f80b-4571-afda-00eb4fe990f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.311433 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19e2e2-f80b-4571-afda-00eb4fe990f1 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.311788 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19e2e2-f80b-4571-afda-00eb4fe990f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.312080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19e2e2-f80b-4571-afda-00eb4fe990f1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.322736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19e2e2-f80b-4571-afda-00eb4fe990f1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.324380 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19e2e2-f80b-4571-afda-00eb4fe990f1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-3d19e2e2-f80b-4571-afda-00eb4fe990f1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:57:14.342299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:14.347039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:14.348354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:14.348354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:14.348354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:57:14.348354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:57:14.348354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:57:14.348354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:57:14.348354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:57:14.348354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:57:14.348354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:57:14.348354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:14.348354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 625/1253] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:14.350216 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982734b-e079-4ee6-b07f-31158bf7e71b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.418583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:14.487701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982734b-e079-4ee6-b07f-31158bf7e71b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.534540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd72e5-17bf-4f15-977b-94b2f0d9d920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.535563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd72e5-17bf-4f15-977b-94b2f0d9d920 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.536045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd72e5-17bf-4f15-977b-94b2f0d9d920 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.536528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd72e5-17bf-4f15-977b-94b2f0d9d920 None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.537656 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd72e5-17bf-4f15-977b-94b2f0d9d920 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:14.539163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd72e5-17bf-4f15-977b-94b2f0d9d920 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.541892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd72e5-17bf-4f15-977b-94b2f0d9d920 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.569653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 629/1254] 10.222.0.75 () {56 vars in 1143 bytes} [Thu Jul 14 10:57:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:14.576449 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19e2e2-f80b-4571-afda-00eb4fe990f1 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.691783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19e2e2-f80b-4571-afda-00eb4fe990f1 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.729359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07c054-8082-4289-8294-2cceee2f6b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.730036 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07c054-8082-4289-8294-2cceee2f6b2d None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.730307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07c054-8082-4289-8294-2cceee2f6b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.730589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07c054-8082-4289-8294-2cceee2f6b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.793989 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:14.811275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07c054-8082-4289-8294-2cceee2f6b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.812815 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07c054-8082-4289-8294-2cceee2f6b2d None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.933953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 626/1255] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:57:14.935292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd72e5-17bf-4f15-977b-94b2f0d9d920 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.047196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd72e5-17bf-4f15-977b-94b2f0d9d920 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.062735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:15.086100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70264f2e-27d1-46fa-b04f-d50f80730d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.087174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70264f2e-27d1-46fa-b04f-d50f80730d7c None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.087491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70264f2e-27d1-46fa-b04f-d50f80730d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.087775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70264f2e-27d1-46fa-b04f-d50f80730d7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.099939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70264f2e-27d1-46fa-b04f-d50f80730d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.101403 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70264f2e-27d1-46fa-b04f-d50f80730d7c None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-70264f2e-27d1-46fa-b04f-d50f80730d7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:15.121928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:15.123083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:15.124410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:15.124410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:15.124410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:15.124410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:15.124410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 630/1256] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:15.129171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07c054-8082-4289-8294-2cceee2f6b2d None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.241009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:15.254624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07c054-8082-4289-8294-2cceee2f6b2d None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.294892 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652f7ca-7205-4804-a5ea-b763850455bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.295618 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652f7ca-7205-4804-a5ea-b763850455bb None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.295871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652f7ca-7205-4804-a5ea-b763850455bb None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.296146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652f7ca-7205-4804-a5ea-b763850455bb None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.296876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652f7ca-7205-4804-a5ea-b763850455bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:15.297250 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652f7ca-7205-4804-a5ea-b763850455bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.298603 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652f7ca-7205-4804-a5ea-b763850455bb None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.311700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 627/1257] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:15.319336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70264f2e-27d1-46fa-b04f-d50f80730d7c None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.434728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70264f2e-27d1-46fa-b04f-d50f80730d7c None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.479717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd45648-f2f3-4f13-bf8b-560f7cf45665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.480529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd45648-f2f3-4f13-bf8b-560f7cf45665 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.480816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd45648-f2f3-4f13-bf8b-560f7cf45665 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.481090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd45648-f2f3-4f13-bf8b-560f7cf45665 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.489703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd45648-f2f3-4f13-bf8b-560f7cf45665 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.491485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd45648-f2f3-4f13-bf8b-560f7cf45665 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-ecd45648-f2f3-4f13-bf8b-560f7cf45665 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:57:15.508931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:15.510074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:15.511215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:57:15.511215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:57:15.511215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:57:15.511215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:57:15.511215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:57:15.511215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:57:15.511215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:57:15.511215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:15.511215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 631/1258] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:57:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:15.516225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652f7ca-7205-4804-a5ea-b763850455bb None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.637394 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652f7ca-7205-4804-a5ea-b763850455bb None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.676741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6c5aa-3653-4090-b3e8-e4f63ce3cd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.677424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6c5aa-3653-4090-b3e8-e4f63ce3cd74 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.677714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6c5aa-3653-4090-b3e8-e4f63ce3cd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.678000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6c5aa-3653-4090-b3e8-e4f63ce3cd74 None admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.678793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6c5aa-3653-4090-b3e8-e4f63ce3cd74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:15.679109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6c5aa-3653-4090-b3e8-e4f63ce3cd74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.680610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6c5aa-3653-4090-b3e8-e4f63ce3cd74 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.694442 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 628/1259] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:57:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:15.700545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd45648-f2f3-4f13-bf8b-560f7cf45665 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.729076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:15.818803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd45648-f2f3-4f13-bf8b-560f7cf45665 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.859747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59312628-b2fe-43ba-983d-42787cdc52ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.860388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59312628-b2fe-43ba-983d-42787cdc52ff None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.860675 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59312628-b2fe-43ba-983d-42787cdc52ff None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.860958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59312628-b2fe-43ba-983d-42787cdc52ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.861811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59312628-b2fe-43ba-983d-42787cdc52ff None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=0fc1f2abce4845219741f4f48bf32d36) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:15.862126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59312628-b2fe-43ba-983d-42787cdc52ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.863560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59312628-b2fe-43ba-983d-42787cdc52ff None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.869384 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 632/1260] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:15 2022] GET /identity/v3/role_assignments?user.id=0fc1f2abce4845219741f4f48bf32d36&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:18.057373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2989e-97e0-4f46-905a-2118bffef176 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:18.058035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2989e-97e0-4f46-905a-2118bffef176 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:18.058035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2989e-97e0-4f46-905a-2118bffef176 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:18.059549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 629/1261] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:18.064748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade784f4-c1da-4843-b9f6-b3c0d3122431 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:18.065180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade784f4-c1da-4843-b9f6-b3c0d3122431 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:18.065320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade784f4-c1da-4843-b9f6-b3c0d3122431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:18.082361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ade784f4-c1da-4843-b9f6-b3c0d3122431 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:18.083599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ade784f4-c1da-4843-b9f6-b3c0d3122431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:18.149715 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 633/1262] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:57:18.172454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2989e-97e0-4f46-905a-2118bffef176 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:18.183974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:18.276370 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2989e-97e0-4f46-905a-2118bffef176 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:18.309701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e32bd4-631c-424c-bde1-74050e7ba610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:18.310410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e32bd4-631c-424c-bde1-74050e7ba610 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:18.310694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e32bd4-631c-424c-bde1-74050e7ba610 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:18.310970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e32bd4-631c-424c-bde1-74050e7ba610 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:18.321493 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e32bd4-631c-424c-bde1-74050e7ba610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-a3e32bd4-631c-424c-bde1-74050e7ba610 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:18.329115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:18.330272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:18.331158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 630/1263] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:57:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:18.336541 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade784f4-c1da-4843-b9f6-b3c0d3122431 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:18.354710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:18.434834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade784f4-c1da-4843-b9f6-b3c0d3122431 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:18.468236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4f30c-322f-4969-a45b-40db2bd82882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:18.468899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4f30c-322f-4969-a45b-40db2bd82882 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:18.469181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4f30c-322f-4969-a45b-40db2bd82882 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:18.469996 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4f30c-322f-4969-a45b-40db2bd82882 None admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:18.470740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4f30c-322f-4969-a45b-40db2bd82882 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:18.471048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4f30c-322f-4969-a45b-40db2bd82882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:18.472450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4f30c-322f-4969-a45b-40db2bd82882 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:18.486044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 634/1264] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:18.491968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e32bd4-631c-424c-bde1-74050e7ba610 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:18.565651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:18.598605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e32bd4-631c-424c-bde1-74050e7ba610 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:18.631438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6704c-9fbd-4ac7-bf13-3b719bcaf3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:18.632231 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6704c-9fbd-4ac7-bf13-3b719bcaf3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:18.632518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6704c-9fbd-4ac7-bf13-3b719bcaf3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:18.632798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6704c-9fbd-4ac7-bf13-3b719bcaf3a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:18.643166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b6704c-9fbd-4ac7-bf13-3b719bcaf3a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:18.644647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b6704c-9fbd-4ac7-bf13-3b719bcaf3a2 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-67b6704c-9fbd-4ac7-bf13-3b719bcaf3a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:18.662111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:18.663257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:18.664568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:18.664568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:18.664568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:18.664568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:18.664568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 631/1265] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:18.669301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4f30c-322f-4969-a45b-40db2bd82882 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:18.750154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:18.769509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4f30c-322f-4969-a45b-40db2bd82882 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:18.803091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c34d-e274-46fb-9abc-eee13ce8cc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:18.803788 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c34d-e274-46fb-9abc-eee13ce8cc54 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:18.804079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c34d-e274-46fb-9abc-eee13ce8cc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:18.804360 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c34d-e274-46fb-9abc-eee13ce8cc54 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:18.805083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72c34d-e274-46fb-9abc-eee13ce8cc54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:18.805399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72c34d-e274-46fb-9abc-eee13ce8cc54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:18.806774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72c34d-e274-46fb-9abc-eee13ce8cc54 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:18.818780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 635/1266] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:18.824582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6704c-9fbd-4ac7-bf13-3b719bcaf3a2 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:18.933428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6704c-9fbd-4ac7-bf13-3b719bcaf3a2 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:18.953105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:18.967446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12fd3f-bb64-4493-9138-1bcd26167b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:18.968259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12fd3f-bb64-4493-9138-1bcd26167b42 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:18.968547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12fd3f-bb64-4493-9138-1bcd26167b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:18.968829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12fd3f-bb64-4493-9138-1bcd26167b42 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:18.976861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12fd3f-bb64-4493-9138-1bcd26167b42 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:18.978319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12fd3f-bb64-4493-9138-1bcd26167b42 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-bc12fd3f-bb64-4493-9138-1bcd26167b42 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:57:18.992386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:18.993522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:18.994679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:18.994679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:18.994679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:57:18.994679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:57:18.994679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:57:18.994679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:57:18.994679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:57:18.994679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:57:18.994679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:57:18.994679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:18.994679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 632/1267] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:18.999464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c34d-e274-46fb-9abc-eee13ce8cc54 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:19.098282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c34d-e274-46fb-9abc-eee13ce8cc54 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:19.131032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d6f4c-05bb-47de-a463-1d001a0fb39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:19.131733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d6f4c-05bb-47de-a463-1d001a0fb39f None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:19.132022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d6f4c-05bb-47de-a463-1d001a0fb39f None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:19.132302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d6f4c-05bb-47de-a463-1d001a0fb39f None admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:19.133133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d6f4c-05bb-47de-a463-1d001a0fb39f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:19.133446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d6f4c-05bb-47de-a463-1d001a0fb39f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:19.134873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d6f4c-05bb-47de-a463-1d001a0fb39f None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:19.144127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:19.153219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 636/1268] 10.222.0.75 () {56 vars in 1143 bytes} [Thu Jul 14 10:57:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:19.159408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12fd3f-bb64-4493-9138-1bcd26167b42 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:19.258286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12fd3f-bb64-4493-9138-1bcd26167b42 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:19.290945 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e3c91-715e-4b28-93b9-2289f08a732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:19.291771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e3c91-715e-4b28-93b9-2289f08a732b None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:19.292057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e3c91-715e-4b28-93b9-2289f08a732b None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:19.292336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e3c91-715e-4b28-93b9-2289f08a732b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:19.302851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e3c91-715e-4b28-93b9-2289f08a732b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:19.304496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e3c91-715e-4b28-93b9-2289f08a732b None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-9b8e3c91-715e-4b28-93b9-2289f08a732b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:19.322847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:19.324013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:19.325304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:19.325304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:19.325304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:19.325304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:19.325304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 633/1269] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:19.330378 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d6f4c-05bb-47de-a463-1d001a0fb39f None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:19.428987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d6f4c-05bb-47de-a463-1d001a0fb39f None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:19.463376 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00356bd1-7a94-4681-955b-7bb0e00a718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:19.464063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00356bd1-7a94-4681-955b-7bb0e00a718b None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:19.464359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00356bd1-7a94-4681-955b-7bb0e00a718b None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:19.464643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00356bd1-7a94-4681-955b-7bb0e00a718b None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:19.465365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00356bd1-7a94-4681-955b-7bb0e00a718b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:19.465677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00356bd1-7a94-4681-955b-7bb0e00a718b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:19.467063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00356bd1-7a94-4681-955b-7bb0e00a718b None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:19.479328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 637/1270] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:19.484951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e3c91-715e-4b28-93b9-2289f08a732b None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:19.516280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:19.584348 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e3c91-715e-4b28-93b9-2289f08a732b None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:19.616875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a2a21-f0f0-47a7-b44d-7f509832ecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:19.617667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a2a21-f0f0-47a7-b44d-7f509832ecf4 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:19.617949 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a2a21-f0f0-47a7-b44d-7f509832ecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:19.618227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a2a21-f0f0-47a7-b44d-7f509832ecf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:19.624718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a2a21-f0f0-47a7-b44d-7f509832ecf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:19.626738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a2a21-f0f0-47a7-b44d-7f509832ecf4 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-880a2a21-f0f0-47a7-b44d-7f509832ecf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:57:19.640849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:19.642165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:19.643556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:57:19.643556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:57:19.643556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:57:19.643556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:57:19.643556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:57:19.643556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:57:19.643556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:57:19.643556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:19.643556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 634/1271] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:57:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:19.647912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00356bd1-7a94-4681-955b-7bb0e00a718b None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:19.708877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:19.753456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00356bd1-7a94-4681-955b-7bb0e00a718b None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:19.786063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aaf1ec-75c5-4f56-860a-d202dc48ccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:19.786737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aaf1ec-75c5-4f56-860a-d202dc48ccf8 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:19.787028 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aaf1ec-75c5-4f56-860a-d202dc48ccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:19.787314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aaf1ec-75c5-4f56-860a-d202dc48ccf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:19.788131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aaf1ec-75c5-4f56-860a-d202dc48ccf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:19.788448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aaf1ec-75c5-4f56-860a-d202dc48ccf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:19.789881 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aaf1ec-75c5-4f56-860a-d202dc48ccf8 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:19.802533 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 638/1272] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:57:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:19.808714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a2a21-f0f0-47a7-b44d-7f509832ecf4 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:19.905674 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:19.918958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a2a21-f0f0-47a7-b44d-7f509832ecf4 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:19.952849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea51c19-9835-45d6-bc51-9a8bfa36f0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:19.953765 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea51c19-9835-45d6-bc51-9a8bfa36f0f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:19.954045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea51c19-9835-45d6-bc51-9a8bfa36f0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:19.954320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea51c19-9835-45d6-bc51-9a8bfa36f0f1 None admin] PATH_INFO: `/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/0fc1f2abce4845219741f4f48bf32d36/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:19.956297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea51c19-9835-45d6-bc51-9a8bfa36f0f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a68601f16d1c49548b6e3af0b9819ba3, user_id=0fc1f2abce4845219741f4f48bf32d36, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:19.957731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea51c19-9835-45d6-bc51-9a8bfa36f0f1 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:19.971438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 635/1273] 10.222.0.75 () {58 vars in 1439 bytes} [Thu Jul 14 10:57:19 2022] PUT /identity/v3/projects/a68601f16d1c49548b6e3af0b9819ba3/users/0fc1f2abce4845219741f4f48bf32d36/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:21.970064 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04639e-9f64-42ad-a596-759db8c18578 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:21.970626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04639e-9f64-42ad-a596-759db8c18578 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:21.970626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04639e-9f64-42ad-a596-759db8c18578 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:21.972034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 639/1274] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:21.977074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136daed-b049-4d78-bed1-c53be411fd94 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:21.977510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136daed-b049-4d78-bed1-c53be411fd94 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:21.977645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136daed-b049-4d78-bed1-c53be411fd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:21.994128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-3136daed-b049-4d78-bed1-c53be411fd94 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:21.995330 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3136daed-b049-4d78-bed1-c53be411fd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:22.061623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 636/1275] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:57:22.085455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04639e-9f64-42ad-a596-759db8c18578 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.189599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04639e-9f64-42ad-a596-759db8c18578 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.208563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:22.226424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da0e81-9350-4964-8618-02d5329c32bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.227144 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da0e81-9350-4964-8618-02d5329c32bb None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.227469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da0e81-9350-4964-8618-02d5329c32bb None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.227755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da0e81-9350-4964-8618-02d5329c32bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.238258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da0e81-9350-4964-8618-02d5329c32bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-98da0e81-9350-4964-8618-02d5329c32bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:57:22.245956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:22.247170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:22.248258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 640/1276] 10.222.0.75 () {56 vars in 1096 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:22.254045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136daed-b049-4d78-bed1-c53be411fd94 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.335374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:22.355542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136daed-b049-4d78-bed1-c53be411fd94 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.391193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81d389-f130-47ed-8f6f-f174a21f03e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.391914 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81d389-f130-47ed-8f6f-f174a21f03e0 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.392202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81d389-f130-47ed-8f6f-f174a21f03e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.392486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81d389-f130-47ed-8f6f-f174a21f03e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.393243 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81d389-f130-47ed-8f6f-f174a21f03e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:22.393553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81d389-f130-47ed-8f6f-f174a21f03e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.394934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81d389-f130-47ed-8f6f-f174a21f03e0 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.404029 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 637/1277] 10.222.0.75 () {56 vars in 1099 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:22.410315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da0e81-9350-4964-8618-02d5329c32bb None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.513300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da0e81-9350-4964-8618-02d5329c32bb None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.547139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1bf0b-0813-4d3f-9a01-e459a9bd5317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.548565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1bf0b-0813-4d3f-9a01-e459a9bd5317 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.548849 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1bf0b-0813-4d3f-9a01-e459a9bd5317 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.549130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1bf0b-0813-4d3f-9a01-e459a9bd5317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.560159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1bf0b-0813-4d3f-9a01-e459a9bd5317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.562344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1bf0b-0813-4d3f-9a01-e459a9bd5317 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-e8e1bf0b-0813-4d3f-9a01-e459a9bd5317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:22.581624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:22.587573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:22.588643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:22.588643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:22.588643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:22.588643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:22.588643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 641/1278] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:22.589130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81d389-f130-47ed-8f6f-f174a21f03e0 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.692919 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81d389-f130-47ed-8f6f-f174a21f03e0 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.725504 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82cbe7-1c74-47e4-8825-c229ae8d736a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.726156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82cbe7-1c74-47e4-8825-c229ae8d736a None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.726442 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82cbe7-1c74-47e4-8825-c229ae8d736a None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.726721 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82cbe7-1c74-47e4-8825-c229ae8d736a None admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.727452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82cbe7-1c74-47e4-8825-c229ae8d736a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:22.727759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82cbe7-1c74-47e4-8825-c229ae8d736a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.729124 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82cbe7-1c74-47e4-8825-c229ae8d736a None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.740364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 638/1279] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:22.745986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1bf0b-0813-4d3f-9a01-e459a9bd5317 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.778595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:22.852520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1bf0b-0813-4d3f-9a01-e459a9bd5317 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.893832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b49e6d-3c4d-4ec9-b178-55424f31fe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.895019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b49e6d-3c4d-4ec9-b178-55424f31fe41 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.895534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b49e6d-3c4d-4ec9-b178-55424f31fe41 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.895946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b49e6d-3c4d-4ec9-b178-55424f31fe41 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.906607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b49e6d-3c4d-4ec9-b178-55424f31fe41 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.909026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b49e6d-3c4d-4ec9-b178-55424f31fe41 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-61b49e6d-3c4d-4ec9-b178-55424f31fe41 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:57:22.924481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:22.926275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:22.927854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:22.927854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:22.927854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:57:22.927854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:57:22.927854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:57:22.927854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:57:22.927854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:57:22.927854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:57:22.927854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:57:22.927854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:22.927854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 642/1280] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:22.932168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82cbe7-1c74-47e4-8825-c229ae8d736a None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.936477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:23.039080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82cbe7-1c74-47e4-8825-c229ae8d736a None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:23.074066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3f331-37a5-40ad-b7f4-e495d7fd9965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:23.074735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3f331-37a5-40ad-b7f4-e495d7fd9965 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:23.075024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3f331-37a5-40ad-b7f4-e495d7fd9965 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:23.075302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3f331-37a5-40ad-b7f4-e495d7fd9965 None admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:23.076162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3f331-37a5-40ad-b7f4-e495d7fd9965 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:23.076473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3f331-37a5-40ad-b7f4-e495d7fd9965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:23.077899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3f331-37a5-40ad-b7f4-e495d7fd9965 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:23.095108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 639/1281] 10.222.0.75 () {56 vars in 1143 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:23.100745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b49e6d-3c4d-4ec9-b178-55424f31fe41 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:23.189736 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:23.200538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b49e6d-3c4d-4ec9-b178-55424f31fe41 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:23.233451 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3889f-8f51-48eb-9ca1-5f494a7ab4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:23.234094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3889f-8f51-48eb-9ca1-5f494a7ab4bd None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:23.234379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3889f-8f51-48eb-9ca1-5f494a7ab4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:23.234655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3889f-8f51-48eb-9ca1-5f494a7ab4bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:23.300068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3889f-8f51-48eb-9ca1-5f494a7ab4bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:23.301444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3889f-8f51-48eb-9ca1-5f494a7ab4bd None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:23.386742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:23.397374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 643/1282] 10.222.0.75 () {58 vars in 1282 bytes} [Thu Jul 14 10:57:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:57:23.403220 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3f331-37a5-40ad-b7f4-e495d7fd9965 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:23.465148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:23.505238 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3f331-37a5-40ad-b7f4-e495d7fd9965 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:23.537427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e444bd-571c-4cab-a874-d705f57081a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:23.538214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e444bd-571c-4cab-a874-d705f57081a9 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:23.538499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e444bd-571c-4cab-a874-d705f57081a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:23.538774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e444bd-571c-4cab-a874-d705f57081a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:23.548932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e444bd-571c-4cab-a874-d705f57081a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:23.550427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e444bd-571c-4cab-a874-d705f57081a9 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-44e444bd-571c-4cab-a874-d705f57081a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:57:23.568136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:23.569160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:23.570120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:57:23.570120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:57:23.570120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:57:23.570120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:23.570120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 640/1283] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 10:57:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:23.575027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3889f-8f51-48eb-9ca1-5f494a7ab4bd None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:23.670048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3889f-8f51-48eb-9ca1-5f494a7ab4bd None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:23.703182 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de895b-3366-4402-9148-d699e3699fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:23.703987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de895b-3366-4402-9148-d699e3699fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:23.704412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de895b-3366-4402-9148-d699e3699fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:23.704824 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de895b-3366-4402-9148-d699e3699fb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:23.705598 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de895b-3366-4402-9148-d699e3699fb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:23.705910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de895b-3366-4402-9148-d699e3699fb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:23.707276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de895b-3366-4402-9148-d699e3699fb1 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:23.718148 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 644/1284] 10.222.0.75 () {56 vars in 1109 bytes} [Thu Jul 14 10:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:23.724290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e444bd-571c-4cab-a874-d705f57081a9 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:23.820509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e444bd-571c-4cab-a874-d705f57081a9 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:23.833625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:23.854555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642c89-e2cc-4f7f-98ea-e0aa6efd3a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:23.855331 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642c89-e2cc-4f7f-98ea-e0aa6efd3a1f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:23.855644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642c89-e2cc-4f7f-98ea-e0aa6efd3a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:23.855921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642c89-e2cc-4f7f-98ea-e0aa6efd3a1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:23.862855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06642c89-e2cc-4f7f-98ea-e0aa6efd3a1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:23.864311 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06642c89-e2cc-4f7f-98ea-e0aa6efd3a1f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application [None req-06642c89-e2cc-4f7f-98ea-e0aa6efd3a1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:57:23.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:23.879595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:23.880971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:57:23.880971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:57:23.880971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:57:23.880971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:57:23.880971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:57:23.880971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:57:23.880971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:57:23.880971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: ERROR keystone.server.flask.application Jul 14 10:57:23.880971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 641/1285] 10.222.0.75 () {56 vars in 1111 bytes} [Thu Jul 14 10:57:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:23.886730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de895b-3366-4402-9148-d699e3699fb1 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:23.941051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:23.989675 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de895b-3366-4402-9148-d699e3699fb1 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.021115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4049f-633b-4ff8-af56-e070b124fca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.021780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4049f-633b-4ff8-af56-e070b124fca3 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.022069 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4049f-633b-4ff8-af56-e070b124fca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.022346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4049f-633b-4ff8-af56-e070b124fca3 None admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.023119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4049f-633b-4ff8-af56-e070b124fca3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:24.023449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4049f-633b-4ff8-af56-e070b124fca3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:24.024867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4049f-633b-4ff8-af56-e070b124fca3 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:24.036979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 645/1286] 10.222.0.75 () {56 vars in 1148 bytes} [Thu Jul 14 10:57:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:24.043120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642c89-e2cc-4f7f-98ea-e0aa6efd3a1f None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:24.145458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642c89-e2cc-4f7f-98ea-e0aa6efd3a1f None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.179365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e071885-42d5-47eb-b592-e7c547fe862f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.180055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e071885-42d5-47eb-b592-e7c547fe862f None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.180340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e071885-42d5-47eb-b592-e7c547fe862f None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.180621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e071885-42d5-47eb-b592-e7c547fe862f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.181449 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e071885-42d5-47eb-b592-e7c547fe862f None admin] RBAC: Adding query filter params (role.id=a0a5d694236542a5826b5a4bc33e1932, scope.project.id=a68601f16d1c49548b6e3af0b9819ba3, user.id=0fc1f2abce4845219741f4f48bf32d36) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:24.181763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e071885-42d5-47eb-b592-e7c547fe862f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:24.183495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e071885-42d5-47eb-b592-e7c547fe862f None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:24.189433 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 642/1287] 10.222.0.75 () {56 vars in 1374 bytes} [Thu Jul 14 10:57:24 2022] GET /identity/v3/role_assignments?user.id=0fc1f2abce4845219741f4f48bf32d36&scope.project.id=a68601f16d1c49548b6e3af0b9819ba3&role.id=a0a5d694236542a5826b5a4bc33e1932 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:26.259133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105f5c0-3244-417a-bd97-6f9bb6ab08e0 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.259739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105f5c0-3244-417a-bd97-6f9bb6ab08e0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.259739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105f5c0-3244-417a-bd97-6f9bb6ab08e0 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.261081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 646/1288] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:26.266731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0f8bf-479e-4f4f-97a0-492bd5c21442 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.267126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0f8bf-479e-4f4f-97a0-492bd5c21442 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.267294 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0f8bf-479e-4f4f-97a0-492bd5c21442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.283787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-87c0f8bf-479e-4f4f-97a0-492bd5c21442 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:26.284990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-87c0f8bf-479e-4f4f-97a0-492bd5c21442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:26.315163 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:26.353108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 643/1289] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:57:26.376723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105f5c0-3244-417a-bd97-6f9bb6ab08e0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:26.477049 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105f5c0-3244-417a-bd97-6f9bb6ab08e0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:26.488717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:26.512468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2706df2-e1be-4580-bfdc-48dd695009a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:26.513269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2706df2-e1be-4580-bfdc-48dd695009a7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.513551 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2706df2-e1be-4580-bfdc-48dd695009a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.513826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2706df2-e1be-4580-bfdc-48dd695009a7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.514471 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2706df2-e1be-4580-bfdc-48dd695009a7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:26.515889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2706df2-e1be-4580-bfdc-48dd695009a7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-a2706df2-e1be-4580-bfdc-48dd695009a7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:26.527633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:26.528890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:26.529787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 647/1290] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Jul 14 10:57:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:26.534855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0f8bf-479e-4f4f-97a0-492bd5c21442 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:26.641460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0f8bf-479e-4f4f-97a0-492bd5c21442 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:26.675155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969c2ce-2d08-4174-9a0a-e831c58c487c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:26.675842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969c2ce-2d08-4174-9a0a-e831c58c487c None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.676129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969c2ce-2d08-4174-9a0a-e831c58c487c None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.676408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969c2ce-2d08-4174-9a0a-e831c58c487c None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.677108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969c2ce-2d08-4174-9a0a-e831c58c487c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:26.677417 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969c2ce-2d08-4174-9a0a-e831c58c487c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:26.678792 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969c2ce-2d08-4174-9a0a-e831c58c487c None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:26.683833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 644/1291] 10.222.0.75 () {56 vars in 1116 bytes} [Thu Jul 14 10:57:26 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:26.689723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2706df2-e1be-4580-bfdc-48dd695009a7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:26.792575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2706df2-e1be-4580-bfdc-48dd695009a7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:26.825548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3bd14-f4a2-44a7-8982-166bd6ea71d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:26.826208 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3bd14-f4a2-44a7-8982-166bd6ea71d6 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:26.826496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3bd14-f4a2-44a7-8982-166bd6ea71d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:26.826778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3bd14-f4a2-44a7-8982-166bd6ea71d6 None admin] PATH_INFO: `/v3/services` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:26.827502 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3bd14-f4a2-44a7-8982-166bd6ea71d6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:26.827815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3bd14-f4a2-44a7-8982-166bd6ea71d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:26.829208 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3bd14-f4a2-44a7-8982-166bd6ea71d6 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:26.833614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 648/1292] 10.222.0.75 () {56 vars in 1116 bytes} [Thu Jul 14 10:57:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:28.862269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bf0b3-0b54-4876-9e17-e67314b6fcc1 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:28.862794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bf0b3-0b54-4876-9e17-e67314b6fcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:28.862915 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bf0b3-0b54-4876-9e17-e67314b6fcc1 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:28.864307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 645/1293] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:28.869903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff14de-a66c-4cc1-868b-d6c9116a21d8 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:28.870343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff14de-a66c-4cc1-868b-d6c9116a21d8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:28.870535 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff14de-a66c-4cc1-868b-d6c9116a21d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:28.888562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d1ff14de-a66c-4cc1-868b-d6c9116a21d8 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:28.889847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d1ff14de-a66c-4cc1-868b-d6c9116a21d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:28.924181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:28.965023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 649/1294] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:57:28.987756 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bf0b3-0b54-4876-9e17-e67314b6fcc1 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:29.018686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:29.087877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bf0b3-0b54-4876-9e17-e67314b6fcc1 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:29.120414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ebbe7-2a8b-4431-835e-4060e85bdcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:29.121051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ebbe7-2a8b-4431-835e-4060e85bdcda None admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.121333 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ebbe7-2a8b-4431-835e-4060e85bdcda None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.121607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ebbe7-2a8b-4431-835e-4060e85bdcda None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.122322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ebbe7-2a8b-4431-835e-4060e85bdcda None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:29.123834 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ebbe7-2a8b-4431-835e-4060e85bdcda None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:29.137071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 646/1295] 10.222.0.75 () {60 vars in 1141 bytes} [Thu Jul 14 10:57:28 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:31.166497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae975c-d5c7-4c09-81a0-74d7ce7db9cd None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.167059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae975c-d5c7-4c09-81a0-74d7ce7db9cd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.167059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae975c-d5c7-4c09-81a0-74d7ce7db9cd None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.168466 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 650/1296] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:31.174048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2408f24-af41-46bf-afb2-7c35d40b9d19 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.174449 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2408f24-af41-46bf-afb2-7c35d40b9d19 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.174614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2408f24-af41-46bf-afb2-7c35d40b9d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.191246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c2408f24-af41-46bf-afb2-7c35d40b9d19 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:31.192486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c2408f24-af41-46bf-afb2-7c35d40b9d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:31.262518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 647/1297] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:31 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:57:31.285289 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae975c-d5c7-4c09-81a0-74d7ce7db9cd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:31.383425 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae975c-d5c7-4c09-81a0-74d7ce7db9cd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:31.416847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c659a-8a79-47d1-a492-a429b0715c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:31.417576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c659a-8a79-47d1-a492-a429b0715c27 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.417858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c659a-8a79-47d1-a492-a429b0715c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.418134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c659a-8a79-47d1-a492-a429b0715c27 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.418785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c659a-8a79-47d1-a492-a429b0715c27 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:31.420190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c659a-8a79-47d1-a492-a429b0715c27 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-641c659a-8a79-47d1-a492-a429b0715c27 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:31.433548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:31.434793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:31.435811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 651/1298] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Jul 14 10:57:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:31.440822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2408f24-af41-46bf-afb2-7c35d40b9d19 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:31.458636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:31.538063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2408f24-af41-46bf-afb2-7c35d40b9d19 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:31.571905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be3e5e-558d-4dbd-a38c-3d28d300a658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:31.572561 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be3e5e-558d-4dbd-a38c-3d28d300a658 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.572847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be3e5e-558d-4dbd-a38c-3d28d300a658 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.573128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be3e5e-558d-4dbd-a38c-3d28d300a658 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.573835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78be3e5e-558d-4dbd-a38c-3d28d300a658 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:31.574144 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78be3e5e-558d-4dbd-a38c-3d28d300a658 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:31.575552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78be3e5e-558d-4dbd-a38c-3d28d300a658 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:31.581065 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 648/1299] 10.222.0.75 () {56 vars in 1116 bytes} [Thu Jul 14 10:57:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:31.586889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c659a-8a79-47d1-a492-a429b0715c27 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:31.617828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:31.684721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c659a-8a79-47d1-a492-a429b0715c27 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:31.716821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22716cba-369c-4de5-bdb3-692241c7c45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:31.717467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22716cba-369c-4de5-bdb3-692241c7c45d None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.717754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22716cba-369c-4de5-bdb3-692241c7c45d None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.718035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22716cba-369c-4de5-bdb3-692241c7c45d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.718818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22716cba-369c-4de5-bdb3-692241c7c45d None admin] RBAC: Adding query filter params (interface=public, service_id=0f1cc3311f234d419848fb7b712aa212, region_id=RegionOne) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:31.719716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22716cba-369c-4de5-bdb3-692241c7c45d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:31.721084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22716cba-369c-4de5-bdb3-692241c7c45d None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:31.726047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 652/1300] 10.222.0.75 () {56 vars in 1251 bytes} [Thu Jul 14 10:57:31 2022] GET /identity/v3/endpoints?service_id=0f1cc3311f234d419848fb7b712aa212&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:31.731662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be3e5e-558d-4dbd-a38c-3d28d300a658 None admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:31.816008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:31.834541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be3e5e-558d-4dbd-a38c-3d28d300a658 None admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:31.868711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07ba2c-d8d1-4cb9-9cc3-de554a588da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:31.869452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07ba2c-d8d1-4cb9-9cc3-de554a588da8 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.869731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07ba2c-d8d1-4cb9-9cc3-de554a588da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.870209 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07ba2c-d8d1-4cb9-9cc3-de554a588da8 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.871051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07ba2c-d8d1-4cb9-9cc3-de554a588da8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:31.872689 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07ba2c-d8d1-4cb9-9cc3-de554a588da8 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:31.881683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 649/1301] 10.222.0.75 () {56 vars in 1087 bytes} [Thu Jul 14 10:57:31 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:33.908829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1be0d-9664-4f0c-960d-1e739688c768 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:33.909393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1be0d-9664-4f0c-960d-1e739688c768 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:33.909393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1be0d-9664-4f0c-960d-1e739688c768 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:33.910760 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 653/1302] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:57:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:33.915911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94b439-9901-49ac-b960-bc4f1cf6a4d6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:33.916332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94b439-9901-49ac-b960-bc4f1cf6a4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:33.916477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94b439-9901-49ac-b960-bc4f1cf6a4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:33.932709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6d94b439-9901-49ac-b960-bc4f1cf6a4d6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:33.934164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6d94b439-9901-49ac-b960-bc4f1cf6a4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:33.997853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 650/1303] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:57:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:57:34.021186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1be0d-9664-4f0c-960d-1e739688c768 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:34.117834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:34.124870 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1be0d-9664-4f0c-960d-1e739688c768 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:34.158549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36d566-0d26-45ab-955a-4cf3c5b9d4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:34.159430 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36d566-0d26-45ab-955a-4cf3c5b9d4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:34.159668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36d566-0d26-45ab-955a-4cf3c5b9d4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:34.159953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36d566-0d26-45ab-955a-4cf3c5b9d4a7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:34.160598 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36d566-0d26-45ab-955a-4cf3c5b9d4a7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:34.161962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36d566-0d26-45ab-955a-4cf3c5b9d4a7 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application [None req-ea36d566-0d26-45ab-955a-4cf3c5b9d4a7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:57:34.173700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:57:34.174857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application with Lock( Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: ERROR keystone.server.flask.application Jul 14 10:57:34.176110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 654/1304] 10.222.0.75 () {56 vars in 1117 bytes} [Thu Jul 14 10:57:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:57:34.180919 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94b439-9901-49ac-b960-bc4f1cf6a4d6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:34.256432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:34.279444 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94b439-9901-49ac-b960-bc4f1cf6a4d6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:34.312278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bd677-5821-466f-82a9-9abe1a5147a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:34.313579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bd677-5821-466f-82a9-9abe1a5147a6 None admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:34.313865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bd677-5821-466f-82a9-9abe1a5147a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:34.314149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bd677-5821-466f-82a9-9abe1a5147a6 None admin] PATH_INFO: `/v3/services` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:34.314847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bd677-5821-466f-82a9-9abe1a5147a6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:34.315156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bd677-5821-466f-82a9-9abe1a5147a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:34.316557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bd677-5821-466f-82a9-9abe1a5147a6 None admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:34.321631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 651/1305] 10.222.0.75 () {56 vars in 1116 bytes} [Thu Jul 14 10:57:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:34.327331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36d566-0d26-45ab-955a-4cf3c5b9d4a7 None admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:34.424572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36d566-0d26-45ab-955a-4cf3c5b9d4a7 None admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:34.457984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e7d05-d80d-4ce0-86bd-cc1cfd0f4f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:34.458640 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e7d05-d80d-4ce0-86bd-cc1cfd0f4f27 None admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:34.458928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e7d05-d80d-4ce0-86bd-cc1cfd0f4f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:34.459206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e7d05-d80d-4ce0-86bd-cc1cfd0f4f27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:34.459970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e7d05-d80d-4ce0-86bd-cc1cfd0f4f27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:34.461341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e7d05-d80d-4ce0-86bd-cc1cfd0f4f27 None admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:34.479180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 655/1306] 10.222.0.75 () {60 vars in 1144 bytes} [Thu Jul 14 10:57:34 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:44.319024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=30 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:44.474901 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:38.292058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2251e2-4985-49eb-a3ba-5a59c05355d1 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.292658 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2251e2-4985-49eb-a3ba-5a59c05355d1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.292658 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2251e2-4985-49eb-a3ba-5a59c05355d1 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.299933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4facd-77f7-4520-8eb7-7a03697e9f25 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.300429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4facd-77f7-4520-8eb7-7a03697e9f25 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.300429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4facd-77f7-4520-8eb7-7a03697e9f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.303940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 652/1307] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:58:38 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:38.475599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1af4facd-77f7-4520-8eb7-7a03697e9f25 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:38.477599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1af4facd-77f7-4520-8eb7-7a03697e9f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:38.683010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 656/1308] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:58:38.726761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c98b58-bd00-4189-9ccc-c674e4d234ef None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.727323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c98b58-bd00-4189-9ccc-c674e4d234ef None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.728332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c98b58-bd00-4189-9ccc-c674e4d234ef None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.729239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 653/1309] 10.222.0.75 () {54 vars in 964 bytes} [Thu Jul 14 10:58:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:38.736464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb6614-d327-4819-8cb5-b1a8e4f2f242 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:38.739367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb6614-d327-4819-8cb5-b1a8e4f2f242 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:38.739367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb6614-d327-4819-8cb5-b1a8e4f2f242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:38.825515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b1eb6614-d327-4819-8cb5-b1a8e4f2f242 None None] MFA Rules not processed for user `dadd485b7cac42f4aa079cab91973c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:38.827225 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b1eb6614-d327-4819-8cb5-b1a8e4f2f242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:38.860451 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:38.931682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 657/1310] 10.222.0.75 () {58 vars in 1061 bytes} [Thu Jul 14 10:58:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:38.941710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c98b58-bd00-4189-9ccc-c674e4d234ef None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:38.943563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-03c98b58-bd00-4189-9ccc-c674e4d234ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:39.074281 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c98b58-bd00-4189-9ccc-c674e4d234ef None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:39.175045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565d89b-e104-45e1-8511-919ae75bea7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:39.175738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565d89b-e104-45e1-8511-919ae75bea7a service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.176068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565d89b-e104-45e1-8511-919ae75bea7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.176357 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565d89b-e104-45e1-8511-919ae75bea7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.178046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f565d89b-e104-45e1-8511-919ae75bea7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:39.249708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:39.320898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565d89b-e104-45e1-8511-919ae75bea7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:39.322447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565d89b-e104-45e1-8511-919ae75bea7a service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:39.344828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 654/1311] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:41.671242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8be67-e992-4b6e-ae81-4058f1bfef2c None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.671840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8be67-e992-4b6e-ae81-4058f1bfef2c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.671840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8be67-e992-4b6e-ae81-4058f1bfef2c None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.673377 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 658/1312] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:41.678513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6870b6-1c9a-4dc4-9d4c-5c536ac98ea5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:41.678978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6870b6-1c9a-4dc4-9d4c-5c536ac98ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:41.679098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6870b6-1c9a-4dc4-9d4c-5c536ac98ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:41.770851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bf6870b6-1c9a-4dc4-9d4c-5c536ac98ea5 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:41.772365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bf6870b6-1c9a-4dc4-9d4c-5c536ac98ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:41.884860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 655/1313] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:41.923778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8be67-e992-4b6e-ae81-4058f1bfef2c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:41.925374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7ad8be67-e992-4b6e-ae81-4058f1bfef2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:42.038511 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8be67-e992-4b6e-ae81-4058f1bfef2c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:42.114307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:42.118816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cb3d3-4d89-44fb-8d24-807f44d7322d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:42.119604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cb3d3-4d89-44fb-8d24-807f44d7322d service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:42.120240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cb3d3-4d89-44fb-8d24-807f44d7322d service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:42.120859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cb3d3-4d89-44fb-8d24-807f44d7322d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:42.123038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-709cb3d3-4d89-44fb-8d24-807f44d7322d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:42.252433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cb3d3-4d89-44fb-8d24-807f44d7322d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:42.253861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cb3d3-4d89-44fb-8d24-807f44d7322d service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:42.273614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 659/1314] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:44.637104 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf789a7-0c79-4b31-a423-8ae17b1795cb None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:44.637632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf789a7-0c79-4b31-a423-8ae17b1795cb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:44.637632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf789a7-0c79-4b31-a423-8ae17b1795cb None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:44.638955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 656/1315] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:44.644170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3bcb8-5a77-46f2-ab20-4d3b32011656 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:44.644545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3bcb8-5a77-46f2-ab20-4d3b32011656 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:44.644736 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3bcb8-5a77-46f2-ab20-4d3b32011656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:44.709168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b4c3bcb8-5a77-46f2-ab20-4d3b32011656 None None] MFA Rules not processed for user `5e6846265bb648f1b7034eadef52b2d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:44.710394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b4c3bcb8-5a77-46f2-ab20-4d3b32011656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:44.762544 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:44.806468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 660/1316] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:44 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:44.845350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf789a7-0c79-4b31-a423-8ae17b1795cb None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:44.846872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4cf789a7-0c79-4b31-a423-8ae17b1795cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:44.856546 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:44.948551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf789a7-0c79-4b31-a423-8ae17b1795cb None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:45.021509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6b343-d708-40fc-a3d5-cf6986526280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:45.022614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6b343-d708-40fc-a3d5-cf6986526280 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.023125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6b343-d708-40fc-a3d5-cf6986526280 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.023654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6b343-d708-40fc-a3d5-cf6986526280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.025988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e4b6b343-d708-40fc-a3d5-cf6986526280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:45.098874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:45.145204 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6b343-d708-40fc-a3d5-cf6986526280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:45.146639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6b343-d708-40fc-a3d5-cf6986526280 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:45.166550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 657/1317] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:44 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:47.752447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb38dd-6580-4a42-9c7f-18e2aaea8d48 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.752987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb38dd-6580-4a42-9c7f-18e2aaea8d48 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.753105 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb38dd-6580-4a42-9c7f-18e2aaea8d48 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.754393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 661/1318] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:47.759582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f17176-77a4-4a64-a1cf-ac4da3e0c24c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.760010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f17176-77a4-4a64-a1cf-ac4da3e0c24c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.760167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f17176-77a4-4a64-a1cf-ac4da3e0c24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.823699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c4f17176-77a4-4a64-a1cf-ac4da3e0c24c None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:47.824914 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c4f17176-77a4-4a64-a1cf-ac4da3e0c24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:47.925365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 658/1319] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:47.962791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb38dd-6580-4a42-9c7f-18e2aaea8d48 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:47.964370 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-78bb38dd-6580-4a42-9c7f-18e2aaea8d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:48.068769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb38dd-6580-4a42-9c7f-18e2aaea8d48 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:48.142599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30340477-d08f-4af0-980a-169177d56162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:48.143261 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30340477-d08f-4af0-980a-169177d56162 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:48.143584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30340477-d08f-4af0-980a-169177d56162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:48.143867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30340477-d08f-4af0-980a-169177d56162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:48.145537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-30340477-d08f-4af0-980a-169177d56162 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:48.153244 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:48.255718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30340477-d08f-4af0-980a-169177d56162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:48.257153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30340477-d08f-4af0-980a-169177d56162 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:48.276323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 662/1320] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:50.958497 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeccf17-d75c-4ae9-92e7-aaea48e686b8 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.959104 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeccf17-d75c-4ae9-92e7-aaea48e686b8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.959104 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeccf17-d75c-4ae9-92e7-aaea48e686b8 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.960486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 659/1321] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:50.965975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063643f-772d-4f30-9418-de17d1a79dcc None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.966450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063643f-772d-4f30-9418-de17d1a79dcc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.966565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063643f-772d-4f30-9418-de17d1a79dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.031556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-4063643f-772d-4f30-9418-de17d1a79dcc None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:51.032768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4063643f-772d-4f30-9418-de17d1a79dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:51.091692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:51.131961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 663/1322] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:58:51.185129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e211727-df2f-442b-b6b6-1075c1ed6272 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.185987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e211727-df2f-442b-b6b6-1075c1ed6272 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.186277 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e211727-df2f-442b-b6b6-1075c1ed6272 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.187789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 660/1323] 10.222.0.75 () {54 vars in 964 bytes} [Thu Jul 14 10:58:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:58:51.195000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4c5b6-bc03-49dc-b9cd-cf1cfc412f4b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.195111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4c5b6-bc03-49dc-b9cd-cf1cfc412f4b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.195432 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4c5b6-bc03-49dc-b9cd-cf1cfc412f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.280554 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-dfb4c5b6-bc03-49dc-b9cd-cf1cfc412f4b None None] MFA Rules not processed for user `dadd485b7cac42f4aa079cab91973c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:51.281721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-dfb4c5b6-bc03-49dc-b9cd-cf1cfc412f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:51.368517 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 664/1324] 10.222.0.75 () {58 vars in 1061 bytes} [Thu Jul 14 10:58:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:51.378115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e211727-df2f-442b-b6b6-1075c1ed6272 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:51.379684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0e211727-df2f-442b-b6b6-1075c1ed6272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:51.399422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:51.501259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e211727-df2f-442b-b6b6-1075c1ed6272 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:51.580710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f2cd7-c982-4271-94e7-8e3444fc93a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:51.581265 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f2cd7-c982-4271-94e7-8e3444fc93a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.581553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f2cd7-c982-4271-94e7-8e3444fc93a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.581835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f2cd7-c982-4271-94e7-8e3444fc93a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.583564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-008f2cd7-c982-4271-94e7-8e3444fc93a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:51.660470 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:51.701930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f2cd7-c982-4271-94e7-8e3444fc93a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:51.705947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f2cd7-c982-4271-94e7-8e3444fc93a0 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:51.723268 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 661/1325] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:54.050848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042ebd-0bf1-4e2f-bded-9defd5ad52ab None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:54.051337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042ebd-0bf1-4e2f-bded-9defd5ad52ab None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:54.051474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042ebd-0bf1-4e2f-bded-9defd5ad52ab None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:54.052857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 665/1326] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:54.058541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a30c7-64b9-4a3c-b703-dce9504ad92a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:54.059032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a30c7-64b9-4a3c-b703-dce9504ad92a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:54.059131 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a30c7-64b9-4a3c-b703-dce9504ad92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:54.127574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-827a30c7-64b9-4a3c-b703-dce9504ad92a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:54.128853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-827a30c7-64b9-4a3c-b703-dce9504ad92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:54.228648 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 662/1327] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:54.264657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad042ebd-0bf1-4e2f-bded-9defd5ad52ab None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:54.266236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ad042ebd-0bf1-4e2f-bded-9defd5ad52ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:54.335212 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:54.379124 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad042ebd-0bf1-4e2f-bded-9defd5ad52ab None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:54.453207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece4135-d0d1-476d-9862-5c0f4a9376ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:54.453858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece4135-d0d1-476d-9862-5c0f4a9376ea service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:54.454143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece4135-d0d1-476d-9862-5c0f4a9376ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:54.454420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece4135-d0d1-476d-9862-5c0f4a9376ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:54.456095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fece4135-d0d1-476d-9862-5c0f4a9376ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:54.563122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:54.565933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece4135-d0d1-476d-9862-5c0f4a9376ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:54.568036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece4135-d0d1-476d-9862-5c0f4a9376ea service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:54.586903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 666/1328] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:56.700544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c8271-eb26-4f05-8ffa-b9beae64c46e None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:56.701184 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c8271-eb26-4f05-8ffa-b9beae64c46e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:56.701184 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c8271-eb26-4f05-8ffa-b9beae64c46e None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:56.702585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 663/1329] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:56.708065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c12a8-747f-4d37-93d5-f5037b2dcf00 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:56.708535 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c12a8-747f-4d37-93d5-f5037b2dcf00 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:56.708639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c12a8-747f-4d37-93d5-f5037b2dcf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:56.775881 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-843c12a8-747f-4d37-93d5-f5037b2dcf00 None None] MFA Rules not processed for user `5e6846265bb648f1b7034eadef52b2d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:56.777132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-843c12a8-747f-4d37-93d5-f5037b2dcf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:56.880237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 667/1330] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:56 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:56.929975 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c8271-eb26-4f05-8ffa-b9beae64c46e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:56.931552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e70c8271-eb26-4f05-8ffa-b9beae64c46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:56.964941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:57.050157 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c8271-eb26-4f05-8ffa-b9beae64c46e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:57.122815 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e710ce3-8b33-4d9c-9354-e6eb218c4a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:57.123509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e710ce3-8b33-4d9c-9354-e6eb218c4a13 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:57.123798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e710ce3-8b33-4d9c-9354-e6eb218c4a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:57.124077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e710ce3-8b33-4d9c-9354-e6eb218c4a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:57.125979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7e710ce3-8b33-4d9c-9354-e6eb218c4a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:57.203860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:57.236204 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e710ce3-8b33-4d9c-9354-e6eb218c4a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:57.237775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e710ce3-8b33-4d9c-9354-e6eb218c4a13 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:57.257051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 664/1331] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:56 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:59.705561 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3dc8b-7810-454c-944d-4b5756f0a818 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:59.706093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3dc8b-7810-454c-944d-4b5756f0a818 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:59.706188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3dc8b-7810-454c-944d-4b5756f0a818 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:59.707571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 668/1332] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:58:59.712646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ce787-4f7a-474f-9a02-86be6f95f36e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:59.713088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ce787-4f7a-474f-9a02-86be6f95f36e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:59.713232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ce787-4f7a-474f-9a02-86be6f95f36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:59.778559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f37ce787-4f7a-474f-9a02-86be6f95f36e None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:59.780220 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f37ce787-4f7a-474f-9a02-86be6f95f36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:59.873747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 665/1333] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:59.910419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3dc8b-7810-454c-944d-4b5756f0a818 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:59.912002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4cf3dc8b-7810-454c-944d-4b5756f0a818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:59.980669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:00.014805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3dc8b-7810-454c-944d-4b5756f0a818 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:00.087193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd46347-66b1-4c3c-9134-3fa264891da1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:00.087805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd46347-66b1-4c3c-9134-3fa264891da1 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:00.088095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd46347-66b1-4c3c-9134-3fa264891da1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:00.088378 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd46347-66b1-4c3c-9134-3fa264891da1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:00.090020 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3dd46347-66b1-4c3c-9134-3fa264891da1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:00.202988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd46347-66b1-4c3c-9134-3fa264891da1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:00.204983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd46347-66b1-4c3c-9134-3fa264891da1 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:00.219886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:00.229722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 669/1334] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:02.328927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08763f9-3a48-4996-b2fe-048751c98eea None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:02.329462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08763f9-3a48-4996-b2fe-048751c98eea None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:02.329462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08763f9-3a48-4996-b2fe-048751c98eea None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:02.330843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 666/1335] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:02.336048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f04c45-5f25-4b5c-9a28-89d2d31f9e9a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:02.336442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f04c45-5f25-4b5c-9a28-89d2d31f9e9a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:02.336611 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f04c45-5f25-4b5c-9a28-89d2d31f9e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:02.400370 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-01f04c45-5f25-4b5c-9a28-89d2d31f9e9a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:02.401627 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-01f04c45-5f25-4b5c-9a28-89d2d31f9e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:02.502807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 670/1336] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:02.542049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08763f9-3a48-4996-b2fe-048751c98eea None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:02.543624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c08763f9-3a48-4996-b2fe-048751c98eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:02.583826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:02.647094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08763f9-3a48-4996-b2fe-048751c98eea None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:02.720396 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0675a-fab7-450d-b99f-c874ef08cc05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:02.720966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0675a-fab7-450d-b99f-c874ef08cc05 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:02.721253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0675a-fab7-450d-b99f-c874ef08cc05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:02.721533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0675a-fab7-450d-b99f-c874ef08cc05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:02.723170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-11f0675a-fab7-450d-b99f-c874ef08cc05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:02.812483 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:02.838389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0675a-fab7-450d-b99f-c874ef08cc05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:02.839876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0675a-fab7-450d-b99f-c874ef08cc05 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:02.859131 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 667/1337] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:05.367719 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a5474-b7c1-4e62-aa06-8f35021010df None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.368351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a5474-b7c1-4e62-aa06-8f35021010df None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.368351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a5474-b7c1-4e62-aa06-8f35021010df None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.369688 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 671/1338] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:05.376105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20294cb-60ba-4d3d-b7b3-149c4e1f58ac None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.376660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20294cb-60ba-4d3d-b7b3-149c4e1f58ac None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.376660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20294cb-60ba-4d3d-b7b3-149c4e1f58ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.444582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c20294cb-60ba-4d3d-b7b3-149c4e1f58ac None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:05.445716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c20294cb-60ba-4d3d-b7b3-149c4e1f58ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:05.543971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 668/1339] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:05.583631 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a5474-b7c1-4e62-aa06-8f35021010df None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:05.585184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-946a5474-b7c1-4e62-aa06-8f35021010df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:05.664031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:05.693314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a5474-b7c1-4e62-aa06-8f35021010df None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:05.770577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a9be9-8850-47dd-beee-13d4a43c627c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:05.771148 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a9be9-8850-47dd-beee-13d4a43c627c service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:05.771461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a9be9-8850-47dd-beee-13d4a43c627c service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:05.771745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a9be9-8850-47dd-beee-13d4a43c627c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:05.773406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-118a9be9-8850-47dd-beee-13d4a43c627c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:05.899614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a9be9-8850-47dd-beee-13d4a43c627c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:05.901067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a9be9-8850-47dd-beee-13d4a43c627c service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:05.923312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 672/1340] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:10.007369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12da10-c12d-4a19-b349-77dae7cd4aea None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:10.008028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12da10-c12d-4a19-b349-77dae7cd4aea None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:10.008028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12da10-c12d-4a19-b349-77dae7cd4aea None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:10.009526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 669/1341] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:10.015519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac356588-51a4-4832-b496-6114abaa0722 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:10.016048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac356588-51a4-4832-b496-6114abaa0722 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:10.016155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac356588-51a4-4832-b496-6114abaa0722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:10.020495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:10.086382 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-ac356588-51a4-4832-b496-6114abaa0722 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:10.087632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ac356588-51a4-4832-b496-6114abaa0722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:10.190910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 673/1342] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:10.228582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12da10-c12d-4a19-b349-77dae7cd4aea None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:10.230120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7d12da10-c12d-4a19-b349-77dae7cd4aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:10.280792 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:10.338886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12da10-c12d-4a19-b349-77dae7cd4aea None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:10.412806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd427c0a-9b0c-4a3d-b6cd-bd1834c9fc29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:10.413568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd427c0a-9b0c-4a3d-b6cd-bd1834c9fc29 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:10.413910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd427c0a-9b0c-4a3d-b6cd-bd1834c9fc29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:10.414250 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd427c0a-9b0c-4a3d-b6cd-bd1834c9fc29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:10.415999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cd427c0a-9b0c-4a3d-b6cd-bd1834c9fc29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:10.516364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:10.534675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd427c0a-9b0c-4a3d-b6cd-bd1834c9fc29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:10.536174 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd427c0a-9b0c-4a3d-b6cd-bd1834c9fc29 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:10.555807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 670/1343] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:13.340622 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58f959-4481-4301-b376-bc5d2fe40d53 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:13.341715 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58f959-4481-4301-b376-bc5d2fe40d53 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:13.341715 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58f959-4481-4301-b376-bc5d2fe40d53 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:13.343082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 674/1344] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:13.348872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd00503-47cc-486f-8553-e06cc916e558 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:13.349375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd00503-47cc-486f-8553-e06cc916e558 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:13.349494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd00503-47cc-486f-8553-e06cc916e558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:13.421507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bfd00503-47cc-486f-8553-e06cc916e558 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:13.422723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bfd00503-47cc-486f-8553-e06cc916e558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:13.522311 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 671/1345] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:13.560187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58f959-4481-4301-b376-bc5d2fe40d53 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:13.561716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bd58f959-4481-4301-b376-bc5d2fe40d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:13.657684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:13.674484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58f959-4481-4301-b376-bc5d2fe40d53 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:13.748669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d98d66-2cd7-4f1d-9e8c-3b909cc30e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:13.749336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d98d66-2cd7-4f1d-9e8c-3b909cc30e39 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:13.749623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d98d66-2cd7-4f1d-9e8c-3b909cc30e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:13.749906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d98d66-2cd7-4f1d-9e8c-3b909cc30e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:13.751604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-22d98d66-2cd7-4f1d-9e8c-3b909cc30e39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:13.860771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d98d66-2cd7-4f1d-9e8c-3b909cc30e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:13.862116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d98d66-2cd7-4f1d-9e8c-3b909cc30e39 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:13.882301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 675/1346] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:18.136350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83a22d-cc64-4045-84f0-b1ddb1b4ed29 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.136896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83a22d-cc64-4045-84f0-b1ddb1b4ed29 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.136896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83a22d-cc64-4045-84f0-b1ddb1b4ed29 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.138665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 672/1347] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:18.144773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179149d-3505-4e72-84dc-0c71e86acfff None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.145406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179149d-3505-4e72-84dc-0c71e86acfff None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.145521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179149d-3505-4e72-84dc-0c71e86acfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.168297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:18.239776 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-7179149d-3505-4e72-84dc-0c71e86acfff None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:18.241041 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7179149d-3505-4e72-84dc-0c71e86acfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:18.352746 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 676/1348] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:59:18.391921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83a22d-cc64-4045-84f0-b1ddb1b4ed29 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:18.393589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9b83a22d-cc64-4045-84f0-b1ddb1b4ed29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:18.471947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:18.526633 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83a22d-cc64-4045-84f0-b1ddb1b4ed29 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:18.608624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692b7ea-5706-478c-8f33-b7c2f8620cb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:18.609279 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692b7ea-5706-478c-8f33-b7c2f8620cb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.609563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692b7ea-5706-478c-8f33-b7c2f8620cb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.609843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692b7ea-5706-478c-8f33-b7c2f8620cb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.611542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f692b7ea-5706-478c-8f33-b7c2f8620cb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:18.721309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:18.725307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692b7ea-5706-478c-8f33-b7c2f8620cb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:18.726988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692b7ea-5706-478c-8f33-b7c2f8620cb5 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:18.748417 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 673/1349] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:23.686795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3751c3-edf6-4b8e-a4f0-8e93744e0c43 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.687246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3751c3-edf6-4b8e-a4f0-8e93744e0c43 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.687512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3751c3-edf6-4b8e-a4f0-8e93744e0c43 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.690154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 677/1350] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 10:59:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:23.694511 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dc1bd-aa70-45ae-a1ec-4245dfebd4a3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:23.694950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dc1bd-aa70-45ae-a1ec-4245dfebd4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:23.695145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dc1bd-aa70-45ae-a1ec-4245dfebd4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:23.801565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-630dc1bd-aa70-45ae-a1ec-4245dfebd4a3 None None] MFA Rules not processed for user `0fc1f2abce4845219741f4f48bf32d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:23.802733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-630dc1bd-aa70-45ae-a1ec-4245dfebd4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:23.959362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 674/1351] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 10:59:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:28.118119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab88007-b4b5-4f60-b512-cdb5ced68575 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:28.118613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab88007-b4b5-4f60-b512-cdb5ced68575 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:28.118957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab88007-b4b5-4f60-b512-cdb5ced68575 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:28.120479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 678/1352] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 10:59:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:28.126299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd21a06-1a72-4e02-8b79-43cfec6b84c2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:28.126716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd21a06-1a72-4e02-8b79-43cfec6b84c2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:28.126878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd21a06-1a72-4e02-8b79-43cfec6b84c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:28.199045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:28.224672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0fd21a06-1a72-4e02-8b79-43cfec6b84c2 None None] MFA Rules not processed for user `0fc1f2abce4845219741f4f48bf32d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:28.226044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0fd21a06-1a72-4e02-8b79-43cfec6b84c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:28.338352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 675/1353] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 10:59:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:28.350377 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=59 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:29.759354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de051d0b-1375-44bf-a670-4f41f793d231 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:29.760006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de051d0b-1375-44bf-a670-4f41f793d231 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:29.760006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de051d0b-1375-44bf-a670-4f41f793d231 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:29.761326 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 679/1354] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 10:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:29.766659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebbf38-7c86-40fb-8ce5-68011f36f36c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:29.767087 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebbf38-7c86-40fb-8ce5-68011f36f36c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:29.767261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebbf38-7c86-40fb-8ce5-68011f36f36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:29.843006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-31ebbf38-7c86-40fb-8ce5-68011f36f36c None None] MFA Rules not processed for user `0fc1f2abce4845219741f4f48bf32d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:29.844242 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-31ebbf38-7c86-40fb-8ce5-68011f36f36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:29.888167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:29.953847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 676/1355] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 10:59:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:33.408893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f7599-490f-4dbb-85d8-35cc58fe9f29 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:33.409441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f7599-490f-4dbb-85d8-35cc58fe9f29 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:33.409742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f7599-490f-4dbb-85d8-35cc58fe9f29 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:33.411562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 680/1356] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:33.417080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cfc93-8a58-4802-bda4-9db5efe18a8a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:33.417548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cfc93-8a58-4802-bda4-9db5efe18a8a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:33.417650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cfc93-8a58-4802-bda4-9db5efe18a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:33.510016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7d6cfc93-8a58-4802-bda4-9db5efe18a8a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:33.510016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7d6cfc93-8a58-4802-bda4-9db5efe18a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:33.619428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 677/1357] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:33.673233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4090049-9eaa-4ba2-98ac-0384be2a8caf None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:33.673785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4090049-9eaa-4ba2-98ac-0384be2a8caf None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:33.674475 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4090049-9eaa-4ba2-98ac-0384be2a8caf None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:33.677238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 681/1358] 10.222.0.75 () {54 vars in 961 bytes} [Thu Jul 14 10:59:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:33.686943 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272d0c9-c37e-4558-93ca-1cb28a262cb7 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:33.687814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272d0c9-c37e-4558-93ca-1cb28a262cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:33.688403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272d0c9-c37e-4558-93ca-1cb28a262cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:33.708492 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:33.780241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7272d0c9-c37e-4558-93ca-1cb28a262cb7 None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:33.781724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7272d0c9-c37e-4558-93ca-1cb28a262cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:33.912972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 678/1359] 10.222.0.75 () {58 vars in 1058 bytes} [Thu Jul 14 10:59:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:59:33.921557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4090049-9eaa-4ba2-98ac-0384be2a8caf None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:33.923064 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f4090049-9eaa-4ba2-98ac-0384be2a8caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:34.079299 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4090049-9eaa-4ba2-98ac-0384be2a8caf None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:34.156470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:34.204263 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d6988-b496-4a41-afc6-244b45c80caa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:34.205044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d6988-b496-4a41-afc6-244b45c80caa service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:34.205397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d6988-b496-4a41-afc6-244b45c80caa service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:34.205826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d6988-b496-4a41-afc6-244b45c80caa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:34.207522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5f7d6988-b496-4a41-afc6-244b45c80caa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:34.305381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d6988-b496-4a41-afc6-244b45c80caa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:34.306994 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d6988-b496-4a41-afc6-244b45c80caa service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:34.326957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 682/1360] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:36.590601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df859e-6133-4b16-a777-a754968cd55d None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:36.591099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df859e-6133-4b16-a777-a754968cd55d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:36.591099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df859e-6133-4b16-a777-a754968cd55d None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:36.592599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 679/1361] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:36.597691 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2e260-198e-4f75-8fa3-bfaebec32909 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:36.598115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2e260-198e-4f75-8fa3-bfaebec32909 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:36.598301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2e260-198e-4f75-8fa3-bfaebec32909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:36.673376 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6ca2e260-198e-4f75-8fa3-bfaebec32909 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:36.674705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6ca2e260-198e-4f75-8fa3-bfaebec32909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:36.715682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:36.787537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 683/1362] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:36.830033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df859e-6133-4b16-a777-a754968cd55d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:36.831732 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-86df859e-6133-4b16-a777-a754968cd55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:36.916408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:37.034660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df859e-6133-4b16-a777-a754968cd55d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:37.204436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8a86a-9f8d-40e6-bcb4-80c454011d7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:37.205332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8a86a-9f8d-40e6-bcb4-80c454011d7c service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:37.205854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8a86a-9f8d-40e6-bcb4-80c454011d7c service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:37.206355 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8a86a-9f8d-40e6-bcb4-80c454011d7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:37.208476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-38c8a86a-9f8d-40e6-bcb4-80c454011d7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:37.236358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:37.306757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8a86a-9f8d-40e6-bcb4-80c454011d7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:37.308660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8a86a-9f8d-40e6-bcb4-80c454011d7c service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:37.328741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 680/1363] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:39.729414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d9f31-65c6-491a-a025-211a0e87ec41 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:39.729989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d9f31-65c6-491a-a025-211a0e87ec41 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:39.729989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d9f31-65c6-491a-a025-211a0e87ec41 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:39.731372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 684/1364] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:39.737161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d083664-b2e5-443c-82f7-1df98e0d6c87 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:39.737610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d083664-b2e5-443c-82f7-1df98e0d6c87 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:39.737704 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d083664-b2e5-443c-82f7-1df98e0d6c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:39.825665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7d083664-b2e5-443c-82f7-1df98e0d6c87 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:39.826884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7d083664-b2e5-443c-82f7-1df98e0d6c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:39.938029 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:39.946856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 681/1365] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:40.002969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946d106-3b47-40f4-a225-2d1617e0c1c5 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:40.003330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946d106-3b47-40f4-a225-2d1617e0c1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:40.003697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946d106-3b47-40f4-a225-2d1617e0c1c5 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:40.005127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 685/1366] 10.222.0.75 () {54 vars in 961 bytes} [Thu Jul 14 10:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:40.011615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd208b4c-bd5d-4d2c-ac6e-e73ed7eb2095 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:40.012011 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd208b4c-bd5d-4d2c-ac6e-e73ed7eb2095 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:40.012380 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd208b4c-bd5d-4d2c-ac6e-e73ed7eb2095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:40.092130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-dd208b4c-bd5d-4d2c-ac6e-e73ed7eb2095 None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:40.093468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-dd208b4c-bd5d-4d2c-ac6e-e73ed7eb2095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:40.250283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 682/1367] 10.222.0.75 () {58 vars in 1058 bytes} [Thu Jul 14 10:59:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:59:40.259191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946d106-3b47-40f4-a225-2d1617e0c1c5 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:40.261081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c946d106-3b47-40f4-a225-2d1617e0c1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:40.428832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946d106-3b47-40f4-a225-2d1617e0c1c5 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:40.461501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:40.584401 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02880748-5589-4fbe-b1dc-c9e38715a7a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:40.585809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880748-5589-4fbe-b1dc-c9e38715a7a4 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:40.585809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880748-5589-4fbe-b1dc-c9e38715a7a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:40.585809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880748-5589-4fbe-b1dc-c9e38715a7a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:40.587716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-02880748-5589-4fbe-b1dc-c9e38715a7a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:40.704907 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02880748-5589-4fbe-b1dc-c9e38715a7a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:40.706740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02880748-5589-4fbe-b1dc-c9e38715a7a4 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:40.727419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 686/1368] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:41.673930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210dd63-c70e-4e31-be04-23f31f6c8364 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:41.674482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210dd63-c70e-4e31-be04-23f31f6c8364 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:41.674482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210dd63-c70e-4e31-be04-23f31f6c8364 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:41.676035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 683/1369] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:41.680925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affa7cb-aae7-4c4b-b16c-875bb1077c4d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:41.681361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affa7cb-aae7-4c4b-b16c-875bb1077c4d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:41.681546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affa7cb-aae7-4c4b-b16c-875bb1077c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:41.713821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:41.749168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0affa7cb-aae7-4c4b-b16c-875bb1077c4d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:41.750371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0affa7cb-aae7-4c4b-b16c-875bb1077c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:41.858110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 687/1370] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:42.204572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0831b6b-a0a8-4191-b49c-e56d29fe355a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:42.204572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0831b6b-a0a8-4191-b49c-e56d29fe355a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:42.204572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0831b6b-a0a8-4191-b49c-e56d29fe355a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:42.204572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 684/1371] 10.222.0.75 () {54 vars in 963 bytes} [Thu Jul 14 10:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:42.212343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f77495-1a5e-46ac-8417-80491497ab1f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:42.212343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f77495-1a5e-46ac-8417-80491497ab1f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:42.212343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f77495-1a5e-46ac-8417-80491497ab1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:42.286610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-50f77495-1a5e-46ac-8417-80491497ab1f None None] MFA Rules not processed for user `80230b1b73944e8ebb6fec3dbe0d5c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:42.287886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-50f77495-1a5e-46ac-8417-80491497ab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:42.342888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:42.375274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 688/1372] 10.222.0.75 () {58 vars in 1060 bytes} [Thu Jul 14 10:59:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:42.382648 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0831b6b-a0a8-4191-b49c-e56d29fe355a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:42.384196 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b0831b6b-a0a8-4191-b49c-e56d29fe355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:42.443448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:42.503670 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0831b6b-a0a8-4191-b49c-e56d29fe355a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:42.591963 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f33647-ea9f-43b7-92d6-8f02f55a7776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:42.592640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f33647-ea9f-43b7-92d6-8f02f55a7776 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:42.592931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f33647-ea9f-43b7-92d6-8f02f55a7776 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:42.593207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f33647-ea9f-43b7-92d6-8f02f55a7776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:42.594928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a9f33647-ea9f-43b7-92d6-8f02f55a7776 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:42.714330 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:42.730405 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f33647-ea9f-43b7-92d6-8f02f55a7776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:42.732935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f33647-ea9f-43b7-92d6-8f02f55a7776 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:42.754644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 685/1373] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:42.905993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f9fd9-618c-45ca-938e-3b1b47bc7747 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:42.906660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f9fd9-618c-45ca-938e-3b1b47bc7747 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:42.906660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f9fd9-618c-45ca-938e-3b1b47bc7747 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:42.911978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 689/1374] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:42.913395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aad9b2-f55f-4f34-8341-1bbe3c2e6c48 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:42.913688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aad9b2-f55f-4f34-8341-1bbe3c2e6c48 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:42.913974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aad9b2-f55f-4f34-8341-1bbe3c2e6c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:42.988474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-89aad9b2-f55f-4f34-8341-1bbe3c2e6c48 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:42.989872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-89aad9b2-f55f-4f34-8341-1bbe3c2e6c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:43.115850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 686/1375] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:59:43.157912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f9fd9-618c-45ca-938e-3b1b47bc7747 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:43.159547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f66f9fd9-618c-45ca-938e-3b1b47bc7747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:43.331665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f9fd9-618c-45ca-938e-3b1b47bc7747 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:43.384157 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:43.453450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769ed2d-bba8-40f2-a710-babbca8eb9a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:43.454123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769ed2d-bba8-40f2-a710-babbca8eb9a5 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:43.454410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769ed2d-bba8-40f2-a710-babbca8eb9a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:43.454691 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769ed2d-bba8-40f2-a710-babbca8eb9a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:43.456509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7769ed2d-bba8-40f2-a710-babbca8eb9a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:43.549542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769ed2d-bba8-40f2-a710-babbca8eb9a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:43.551153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769ed2d-bba8-40f2-a710-babbca8eb9a5 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:43.572394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 690/1376] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:45.643783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e7f17-c954-4171-a51f-648c0defd2c8 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:45.644356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e7f17-c954-4171-a51f-648c0defd2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:45.644356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e7f17-c954-4171-a51f-648c0defd2c8 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:45.645699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 687/1377] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:45.651750 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070d126-6abf-400e-93cd-c6e33da6d4b6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:45.652212 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070d126-6abf-400e-93cd-c6e33da6d4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:45.652985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070d126-6abf-400e-93cd-c6e33da6d4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:45.727170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e070d126-6abf-400e-93cd-c6e33da6d4b6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:45.728431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e070d126-6abf-400e-93cd-c6e33da6d4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:45.735776 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:45.850109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 691/1378] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:59:45.896591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e7f17-c954-4171-a51f-648c0defd2c8 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:45.898121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-206e7f17-c954-4171-a51f-648c0defd2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:45.973317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:46.060822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e7f17-c954-4171-a51f-648c0defd2c8 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:46.178533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31737e-7900-44a1-9575-5acfe3f2d811 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:46.178533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31737e-7900-44a1-9575-5acfe3f2d811 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:46.178533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31737e-7900-44a1-9575-5acfe3f2d811 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:46.180392 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31737e-7900-44a1-9575-5acfe3f2d811 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:46.180591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ee31737e-7900-44a1-9575-5acfe3f2d811 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:46.217482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:46.331883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31737e-7900-44a1-9575-5acfe3f2d811 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:46.334290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31737e-7900-44a1-9575-5acfe3f2d811 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:46.357929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 688/1379] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:48.471834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88526977-8950-413b-8be6-dabbaba945ee None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:48.472468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88526977-8950-413b-8be6-dabbaba945ee None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:48.472468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88526977-8950-413b-8be6-dabbaba945ee None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:48.473937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 692/1380] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:48.479285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c91a8-1ce7-4a70-b880-1054859b29da None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:48.479792 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c91a8-1ce7-4a70-b880-1054859b29da None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:48.479885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c91a8-1ce7-4a70-b880-1054859b29da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:48.549452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-889c91a8-1ce7-4a70-b880-1054859b29da None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:48.550873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-889c91a8-1ce7-4a70-b880-1054859b29da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:48.652745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:48.661140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 689/1381] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:48.705167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88526977-8950-413b-8be6-dabbaba945ee None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:48.706687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-88526977-8950-413b-8be6-dabbaba945ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:48.864696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:48.878575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88526977-8950-413b-8be6-dabbaba945ee None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:49.023108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a418257-38ae-4264-bdf8-d4ef2ece7ebd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:49.023897 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a418257-38ae-4264-bdf8-d4ef2ece7ebd service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:49.024223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a418257-38ae-4264-bdf8-d4ef2ece7ebd service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:49.024539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a418257-38ae-4264-bdf8-d4ef2ece7ebd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:49.026440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2a418257-38ae-4264-bdf8-d4ef2ece7ebd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:49.133371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a418257-38ae-4264-bdf8-d4ef2ece7ebd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:49.135117 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a418257-38ae-4264-bdf8-d4ef2ece7ebd service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:49.146956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:49.157730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 693/1382] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:51.514088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9dd11-6b0e-4d83-a52b-96ba6fd6edfc None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:51.514710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9dd11-6b0e-4d83-a52b-96ba6fd6edfc None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:51.514710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9dd11-6b0e-4d83-a52b-96ba6fd6edfc None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:51.516052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 690/1383] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:51.522773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a4a2f-5684-46c5-8b8e-8c592ab437b3 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:51.523275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a4a2f-5684-46c5-8b8e-8c592ab437b3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:51.523590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a4a2f-5684-46c5-8b8e-8c592ab437b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:51.601462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a36a4a2f-5684-46c5-8b8e-8c592ab437b3 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:51.602681 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a36a4a2f-5684-46c5-8b8e-8c592ab437b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:51.711024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 694/1384] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:51.755147 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9dd11-6b0e-4d83-a52b-96ba6fd6edfc None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:51.756701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-49f9dd11-6b0e-4d83-a52b-96ba6fd6edfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:51.918494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9dd11-6b0e-4d83-a52b-96ba6fd6edfc None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:51.992829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:52.041827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e00c8-7573-4080-be5a-1a04dd37c873 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:52.042508 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e00c8-7573-4080-be5a-1a04dd37c873 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:52.043442 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e00c8-7573-4080-be5a-1a04dd37c873 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:52.043689 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e00c8-7573-4080-be5a-1a04dd37c873 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:52.045271 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-753e00c8-7573-4080-be5a-1a04dd37c873 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:52.146274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e00c8-7573-4080-be5a-1a04dd37c873 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:52.147989 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e00c8-7573-4080-be5a-1a04dd37c873 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:52.167610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 691/1385] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:54.612978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb3be5-759d-4822-ab8e-f8ed2e7fc098 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:54.613510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb3be5-759d-4822-ab8e-f8ed2e7fc098 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:54.613510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb3be5-759d-4822-ab8e-f8ed2e7fc098 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:54.614841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 695/1386] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:54.620389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7a037-a750-4678-9df1-bb0f382ebb56 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:54.621385 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7a037-a750-4678-9df1-bb0f382ebb56 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:54.621487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7a037-a750-4678-9df1-bb0f382ebb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:54.690110 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9be7a037-a750-4678-9df1-bb0f382ebb56 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:54.691709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9be7a037-a750-4678-9df1-bb0f382ebb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:54.731258 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:54.806306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 692/1387] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:54.852781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb3be5-759d-4822-ab8e-f8ed2e7fc098 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:54.854350 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cbdb3be5-759d-4822-ab8e-f8ed2e7fc098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:54.941559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:55.022198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb3be5-759d-4822-ab8e-f8ed2e7fc098 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:55.155753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b91dd8-6a5d-412d-9ec1-825717035eeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:55.156407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b91dd8-6a5d-412d-9ec1-825717035eeb service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:55.156668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b91dd8-6a5d-412d-9ec1-825717035eeb service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:55.156952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b91dd8-6a5d-412d-9ec1-825717035eeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:55.158716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-93b91dd8-6a5d-412d-9ec1-825717035eeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:55.201745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:55.262000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b91dd8-6a5d-412d-9ec1-825717035eeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:55.264246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b91dd8-6a5d-412d-9ec1-825717035eeb service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:55.284770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 696/1388] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:57.385010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ab64c-3285-4188-a6a2-0410fb56a8bc None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:57.385578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ab64c-3285-4188-a6a2-0410fb56a8bc None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:57.385578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ab64c-3285-4188-a6a2-0410fb56a8bc None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:57.387565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 693/1389] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:57.392896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725030-48ae-402e-a95e-e5ba8b06ec59 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:57.393333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725030-48ae-402e-a95e-e5ba8b06ec59 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:57.393459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725030-48ae-402e-a95e-e5ba8b06ec59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:57.461792 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3b725030-48ae-402e-a95e-e5ba8b06ec59 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:57.462948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3b725030-48ae-402e-a95e-e5ba8b06ec59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:57.565137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 697/1390] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:57.609121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ab64c-3285-4188-a6a2-0410fb56a8bc None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:57.610650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4b9ab64c-3285-4188-a6a2-0410fb56a8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:57.769137 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ab64c-3285-4188-a6a2-0410fb56a8bc None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:57.786550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:57.892980 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9dfc66-4a7e-42f2-b161-0e90714c0d25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:57.893623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dfc66-4a7e-42f2-b161-0e90714c0d25 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:57.893907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dfc66-4a7e-42f2-b161-0e90714c0d25 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:57.894179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dfc66-4a7e-42f2-b161-0e90714c0d25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:57.895844 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ab9dfc66-4a7e-42f2-b161-0e90714c0d25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:58.018285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9dfc66-4a7e-42f2-b161-0e90714c0d25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:58.020050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9dfc66-4a7e-42f2-b161-0e90714c0d25 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:58.039978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 694/1391] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:58.425803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8c7e8-58f3-457b-a244-4a3cc2b27c61 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:58.426485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8c7e8-58f3-457b-a244-4a3cc2b27c61 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:58.426485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8c7e8-58f3-457b-a244-4a3cc2b27c61 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:58.427817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 698/1392] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 10:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:59:58.434181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587c190-6f6a-4e71-94a8-b298f52033e3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:58.434818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587c190-6f6a-4e71-94a8-b298f52033e3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:58.434818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587c190-6f6a-4e71-94a8-b298f52033e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:58.464125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:58.512096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a587c190-6f6a-4e71-94a8-b298f52033e3 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:58.513328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a587c190-6f6a-4e71-94a8-b298f52033e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:58.614910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 695/1393] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 10:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:58.672419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445c22d-58cf-4bf6-9c39-451508766915 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:58.672975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445c22d-58cf-4bf6-9c39-451508766915 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:58.673478 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445c22d-58cf-4bf6-9c39-451508766915 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:58.676026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 699/1394] 10.222.0.75 () {54 vars in 962 bytes} [Thu Jul 14 10:59:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:58.681317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcaf2c-b957-4075-a17c-a751329ab4f0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:58.681697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcaf2c-b957-4075-a17c-a751329ab4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:58.681932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcaf2c-b957-4075-a17c-a751329ab4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:58.768890 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b4dcaf2c-b957-4075-a17c-a751329ab4f0 None None] MFA Rules not processed for user `d1ad1ffe4d99420cb568d6154f779838`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:58.770116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b4dcaf2c-b957-4075-a17c-a751329ab4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:58.817638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:58.856984 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 696/1395] 10.222.0.75 () {58 vars in 1059 bytes} [Thu Jul 14 10:59:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:59:58.864726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445c22d-58cf-4bf6-9c39-451508766915 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:58.866256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a445c22d-58cf-4bf6-9c39-451508766915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:58.881522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:58.984197 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445c22d-58cf-4bf6-9c39-451508766915 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:59.063978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b68204-f933-44d7-96b9-59adac37f261 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1ad1ffe4d99420cb568d6154f779838', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:59.064645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b68204-f933-44d7-96b9-59adac37f261 service glance] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:59.064919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b68204-f933-44d7-96b9-59adac37f261 service glance] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:59.065194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b68204-f933-44d7-96b9-59adac37f261 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:59.066855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e5b68204-f933-44d7-96b9-59adac37f261 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:59.262998 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:59.318761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b68204-f933-44d7-96b9-59adac37f261 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:59.320253 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b68204-f933-44d7-96b9-59adac37f261 service glance] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:59.339223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 700/1396] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:00.065183 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d9b8b-d461-4a23-bfba-8d755f392c16 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:00.065724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d9b8b-d461-4a23-bfba-8d755f392c16 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:00.066162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d9b8b-d461-4a23-bfba-8d755f392c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:00.165179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b97d9b8b-d461-4a23-bfba-8d755f392c16 None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:00.166390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b97d9b8b-d461-4a23-bfba-8d755f392c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:00.192017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09180d74-da8c-4789-8870-8f0d73331d44 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:00.192017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09180d74-da8c-4789-8870-8f0d73331d44 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:00.192017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09180d74-da8c-4789-8870-8f0d73331d44 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:00.192017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 701/1397] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 11:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:00.196210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52551745-a870-4dc5-bb6f-08dd992df060 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:00.196383 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52551745-a870-4dc5-bb6f-08dd992df060 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:00.196663 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52551745-a870-4dc5-bb6f-08dd992df060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:00.270098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 697/1398] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:00:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:00.276741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-52551745-a870-4dc5-bb6f-08dd992df060 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:00.278067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-52551745-a870-4dc5-bb6f-08dd992df060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:00.320845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a464dc-a12b-4639-acd9-1626cce88f37 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:00.321140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a464dc-a12b-4639-acd9-1626cce88f37 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:00.321428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a464dc-a12b-4639-acd9-1626cce88f37 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:00.322963 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 698/1399] 10.222.0.75 () {54 vars in 942 bytes} [Thu Jul 14 11:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:00:00.329516 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08843cd-9523-4894-94f3-3eb3cff8badf None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:00.329800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08843cd-9523-4894-94f3-3eb3cff8badf None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:00.330072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08843cd-9523-4894-94f3-3eb3cff8badf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:00.385958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 702/1400] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:00:00.405574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:00.418398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d08843cd-9523-4894-94f3-3eb3cff8badf None None] MFA Rules not processed for user `978e71dfbb134f41871114824b03fbeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:00.420070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d08843cd-9523-4894-94f3-3eb3cff8badf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:00.433766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52551745-a870-4dc5-bb6f-08dd992df060 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:00.435150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-52551745-a870-4dc5-bb6f-08dd992df060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:00.463348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:00.545207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 699/1401] 10.222.0.75 () {58 vars in 1039 bytes} [Thu Jul 14 11:00:00 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:00:00.554917 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08843cd-9523-4894-94f3-3eb3cff8badf None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:00.565196 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08843cd-9523-4894-94f3-3eb3cff8badf None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:00.570490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52551745-a870-4dc5-bb6f-08dd992df060 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:00.692252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e0d4e-1add-431d-95b1-d3f8a9dbddfd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:00.692672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e0d4e-1add-431d-95b1-d3f8a9dbddfd service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:00.692989 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e0d4e-1add-431d-95b1-d3f8a9dbddfd service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:00.693283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e0d4e-1add-431d-95b1-d3f8a9dbddfd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:00.695306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f73e0d4e-1add-431d-95b1-d3f8a9dbddfd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:00.698040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:00.699427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5fd39-a5ab-45e0-9fad-9070020a9ff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:00.699427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5fd39-a5ab-45e0-9fad-9070020a9ff1 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:00.699716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5fd39-a5ab-45e0-9fad-9070020a9ff1 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:00.699983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5fd39-a5ab-45e0-9fad-9070020a9ff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:00.701755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-16d5fd39-a5ab-45e0-9fad-9070020a9ff1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:00.721522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:00.795461 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e0d4e-1add-431d-95b1-d3f8a9dbddfd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:00.797102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e0d4e-1add-431d-95b1-d3f8a9dbddfd service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:00.805877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 700/1402] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:00:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 11:00:00.807128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5fd39-a5ab-45e0-9fad-9070020a9ff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:00.809087 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5fd39-a5ab-45e0-9fad-9070020a9ff1 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:00.828896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 703/1403] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:00:02.974454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad740b-6139-4c8e-bcb7-953deaaf76cd None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:02.975079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad740b-6139-4c8e-bcb7-953deaaf76cd None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:02.975079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad740b-6139-4c8e-bcb7-953deaaf76cd None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:02.977872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 701/1404] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 11:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:02.981800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b726c-2a6f-40cb-84d8-03e6aa40c1f9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:02.982301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b726c-2a6f-40cb-84d8-03e6aa40c1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:02.982410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b726c-2a6f-40cb-84d8-03e6aa40c1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:03.074714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a82b726c-2a6f-40cb-84d8-03e6aa40c1f9 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:03.076372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a82b726c-2a6f-40cb-84d8-03e6aa40c1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:03.195682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:03.207456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 704/1405] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:00:03.250341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad740b-6139-4c8e-bcb7-953deaaf76cd None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:03.251902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-afad740b-6139-4c8e-bcb7-953deaaf76cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:03.418354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:03.421318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad740b-6139-4c8e-bcb7-953deaaf76cd None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:03.545745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f013a-303e-4b9b-a33e-848de63bdf98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:03.546479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f013a-303e-4b9b-a33e-848de63bdf98 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:03.546768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f013a-303e-4b9b-a33e-848de63bdf98 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:03.547048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f013a-303e-4b9b-a33e-848de63bdf98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:03.548771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7d9f013a-303e-4b9b-a33e-848de63bdf98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:03.647288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f013a-303e-4b9b-a33e-848de63bdf98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:03.648935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f013a-303e-4b9b-a33e-848de63bdf98 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:03.670632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 702/1406] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:05.790709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546cafb-5410-4500-85b5-af8d8f769cf0 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:05.790709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546cafb-5410-4500-85b5-af8d8f769cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:05.790709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546cafb-5410-4500-85b5-af8d8f769cf0 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:05.790709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 705/1407] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 11:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:05.791924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc670b-ce76-4680-bf8a-71a4f91b2980 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:05.792307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc670b-ce76-4680-bf8a-71a4f91b2980 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:05.792547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc670b-ce76-4680-bf8a-71a4f91b2980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:05.801774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:05.905810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b5dc670b-ce76-4680-bf8a-71a4f91b2980 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:05.907496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b5dc670b-ce76-4680-bf8a-71a4f91b2980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:06.043644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 703/1408] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 11:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:06.086292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546cafb-5410-4500-85b5-af8d8f769cf0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:06.086652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6546cafb-5410-4500-85b5-af8d8f769cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:06.286534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546cafb-5410-4500-85b5-af8d8f769cf0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:06.362809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:06.419393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64525bf-40f6-42c9-aff3-23aabc01dd34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:06.420078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64525bf-40f6-42c9-aff3-23aabc01dd34 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:06.420364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64525bf-40f6-42c9-aff3-23aabc01dd34 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:06.420639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64525bf-40f6-42c9-aff3-23aabc01dd34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:06.422272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d64525bf-40f6-42c9-aff3-23aabc01dd34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:06.523735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64525bf-40f6-42c9-aff3-23aabc01dd34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:06.525347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64525bf-40f6-42c9-aff3-23aabc01dd34 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:06.546260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 706/1409] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:16.036914 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=64 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:16.543047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=64 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:23.381238 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6efd17-979d-44ae-9ac0-022793545e6a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:23.381849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6efd17-979d-44ae-9ac0-022793545e6a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:23.381849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6efd17-979d-44ae-9ac0-022793545e6a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:23.383198 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 704/1410] 10.222.0.75 () {54 vars in 918 bytes} [Thu Jul 14 11:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:23.388725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd2909-af29-4015-b43a-e81a822d11f9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:23.389146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd2909-af29-4015-b43a-e81a822d11f9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:23.389281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd2909-af29-4015-b43a-e81a822d11f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:23.462561 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f6bd2909-af29-4015-b43a-e81a822d11f9 None None] MFA Rules not processed for user `0fc1f2abce4845219741f4f48bf32d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:23.463723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f6bd2909-af29-4015-b43a-e81a822d11f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:23.569270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 707/1411] 10.222.0.75 () {58 vars in 1015 bytes} [Thu Jul 14 11:00:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:23.583928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10ee78-5d4f-4209-99f6-bde36a01b361 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:23.584248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10ee78-5d4f-4209-99f6-bde36a01b361 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:23.584533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10ee78-5d4f-4209-99f6-bde36a01b361 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:23.585882 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 705/1412] 10.222.0.75 () {54 vars in 918 bytes} [Thu Jul 14 11:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:23.591319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e71f14-d4f8-4c4c-9c5c-cfcfa8d73802 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:23.591642 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e71f14-d4f8-4c4c-9c5c-cfcfa8d73802 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:23.591929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e71f14-d4f8-4c4c-9c5c-cfcfa8d73802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:23.665002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:23.667687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e1e71f14-d4f8-4c4c-9c5c-cfcfa8d73802 None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:23.669536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e1e71f14-d4f8-4c4c-9c5c-cfcfa8d73802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:23.785284 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 708/1413] 10.222.0.75 () {58 vars in 1015 bytes} [Thu Jul 14 11:00:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:00:26.285786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dd70e-2d67-41c5-84ae-2be0b05819a6 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:26.286312 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dd70e-2d67-41c5-84ae-2be0b05819a6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:26.286420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dd70e-2d67-41c5-84ae-2be0b05819a6 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:26.287767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:26.293040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810be44-c100-4dd2-9623-957f98813cf9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:26.294004 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810be44-c100-4dd2-9623-957f98813cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:26.294314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810be44-c100-4dd2-9623-957f98813cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:26.364043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d810be44-c100-4dd2-9623-957f98813cf9 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:26.365435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d810be44-c100-4dd2-9623-957f98813cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:26.421712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:26.464139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:26.507613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dd70e-2d67-41c5-84ae-2be0b05819a6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:26.509183 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-671dd70e-2d67-41c5-84ae-2be0b05819a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:26.652035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dd70e-2d67-41c5-84ae-2be0b05819a6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:26.723230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:26.763151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3d428-1597-4790-95ba-8153130ac156 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:26.763828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3d428-1597-4790-95ba-8153130ac156 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:26.764115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3d428-1597-4790-95ba-8153130ac156 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:26.764388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3d428-1597-4790-95ba-8153130ac156 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:26.766025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-62b3d428-1597-4790-95ba-8153130ac156 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:26.853014 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3d428-1597-4790-95ba-8153130ac156 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:26.854632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3d428-1597-4790-95ba-8153130ac156 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:26.873141 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 707/1416] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:27.778390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bcc22-bf39-4b8b-884c-6d2818b525f2 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:27.778965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bcc22-bf39-4b8b-884c-6d2818b525f2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:27.778965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bcc22-bf39-4b8b-884c-6d2818b525f2 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:27.780381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:27.785346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03714d62-d72e-43e2-ab19-76c5e4cad782 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:27.785780 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03714d62-d72e-43e2-ab19-76c5e4cad782 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:27.785890 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03714d62-d72e-43e2-ab19-76c5e4cad782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:27.859251 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-03714d62-d72e-43e2-ab19-76c5e4cad782 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:27.860538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-03714d62-d72e-43e2-ab19-76c5e4cad782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:27.887716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:27.962464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:28.112241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bcc22-bf39-4b8b-884c-6d2818b525f2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:28.113673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5a3bcc22-bf39-4b8b-884c-6d2818b525f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:28.260343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bcc22-bf39-4b8b-884c-6d2818b525f2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:28.300445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:28.379523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256154c-2260-4d72-86c3-2c1e0c960f43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:28.380193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256154c-2260-4d72-86c3-2c1e0c960f43 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:28.380500 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256154c-2260-4d72-86c3-2c1e0c960f43 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:28.380785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256154c-2260-4d72-86c3-2c1e0c960f43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:28.382535 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8256154c-2260-4d72-86c3-2c1e0c960f43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:28.472164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256154c-2260-4d72-86c3-2c1e0c960f43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:28.473761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256154c-2260-4d72-86c3-2c1e0c960f43 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:28.492152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 711/1419] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:29.390299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424343a-4f49-4fe0-9745-1f69d153df8a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:29.390867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424343a-4f49-4fe0-9745-1f69d153df8a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:29.390867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424343a-4f49-4fe0-9745-1f69d153df8a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:29.392315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:29.397071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda4808-2524-4796-a7fa-a622726cadd5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:29.397459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda4808-2524-4796-a7fa-a622726cadd5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:29.397631 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda4808-2524-4796-a7fa-a622726cadd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:29.449463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:29.470699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3dda4808-2524-4796-a7fa-a622726cadd5 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:29.471923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3dda4808-2524-4796-a7fa-a622726cadd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:29.568851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:29.720340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424343a-4f49-4fe0-9745-1f69d153df8a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:29.721801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1424343a-4f49-4fe0-9745-1f69d153df8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:29.865705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424343a-4f49-4fe0-9745-1f69d153df8a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:29.877729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:29.977526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24938f-0183-4f98-a4f5-bb39676bf945 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:29.978086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24938f-0183-4f98-a4f5-bb39676bf945 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:29.978372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24938f-0183-4f98-a4f5-bb39676bf945 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:29.978651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24938f-0183-4f98-a4f5-bb39676bf945 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:29.980346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ec24938f-0183-4f98-a4f5-bb39676bf945 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:30.075023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24938f-0183-4f98-a4f5-bb39676bf945 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:30.077117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24938f-0183-4f98-a4f5-bb39676bf945 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:30.096152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 710/1422] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:30.978729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c0ddf-7fdf-44e3-abf5-c08abebe738d None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:30.979330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c0ddf-7fdf-44e3-abf5-c08abebe738d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:30.979330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c0ddf-7fdf-44e3-abf5-c08abebe738d None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:30.981279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:00:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:30.986248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bdb4b-7bb4-4dc5-bc20-dd328fd290e3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:30.986683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bdb4b-7bb4-4dc5-bc20-dd328fd290e3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:30.986813 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bdb4b-7bb4-4dc5-bc20-dd328fd290e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:31.008534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:31.084389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f52bdb4b-7bb4-4dc5-bc20-dd328fd290e3 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:31.085614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f52bdb4b-7bb4-4dc5-bc20-dd328fd290e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:31.181397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:31.332903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c0ddf-7fdf-44e3-abf5-c08abebe738d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:31.334323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-295c0ddf-7fdf-44e3-abf5-c08abebe738d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:31.453180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:31.489375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c0ddf-7fdf-44e3-abf5-c08abebe738d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:31.601210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084170ea-20a3-430d-b353-7926ab61057c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:31.601772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084170ea-20a3-430d-b353-7926ab61057c service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:31.602055 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084170ea-20a3-430d-b353-7926ab61057c service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:31.602333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084170ea-20a3-430d-b353-7926ab61057c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:31.603989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-084170ea-20a3-430d-b353-7926ab61057c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:31.696873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:31.710528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084170ea-20a3-430d-b353-7926ab61057c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:31.712148 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084170ea-20a3-430d-b353-7926ab61057c service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:31.729853 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 714/1425] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:32.611937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d3afe-8725-4677-903c-1345ce8dce52 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:32.612457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d3afe-8725-4677-903c-1345ce8dce52 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:32.612457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d3afe-8725-4677-903c-1345ce8dce52 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:32.613795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:32.618716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794b277-3b0a-4f99-8c21-12f9ef878c16 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:32.619123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794b277-3b0a-4f99-8c21-12f9ef878c16 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:32.619286 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794b277-3b0a-4f99-8c21-12f9ef878c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:32.696698 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-7794b277-3b0a-4f99-8c21-12f9ef878c16 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:32.697934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7794b277-3b0a-4f99-8c21-12f9ef878c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:32.793819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:32.943038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d3afe-8725-4677-903c-1345ce8dce52 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:32.944589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-873d3afe-8725-4677-903c-1345ce8dce52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:33.044268 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:33.106501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d3afe-8725-4677-903c-1345ce8dce52 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:33.220361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d32816-5f22-456a-8f70-928e18f9a8c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:33.221021 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d32816-5f22-456a-8f70-928e18f9a8c9 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:33.221306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d32816-5f22-456a-8f70-928e18f9a8c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:33.221583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d32816-5f22-456a-8f70-928e18f9a8c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:33.223244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f0d32816-5f22-456a-8f70-928e18f9a8c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:33.271418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:33.313507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d32816-5f22-456a-8f70-928e18f9a8c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:33.315121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d32816-5f22-456a-8f70-928e18f9a8c9 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:33.334234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 713/1428] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:34.158917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf8fd3-05aa-4b53-8849-0631ad8ee3dc None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:34.159605 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf8fd3-05aa-4b53-8849-0631ad8ee3dc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:34.159605 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf8fd3-05aa-4b53-8849-0631ad8ee3dc None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:34.160991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:34.165966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36da355-a84a-4b9f-9a4f-4a7b453fd1ee None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:34.166397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36da355-a84a-4b9f-9a4f-4a7b453fd1ee None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:34.166556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36da355-a84a-4b9f-9a4f-4a7b453fd1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:34.243976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a36da355-a84a-4b9f-9a4f-4a7b453fd1ee None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:34.245194 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a36da355-a84a-4b9f-9a4f-4a7b453fd1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:34.342577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:34.520806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf8fd3-05aa-4b53-8849-0631ad8ee3dc None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:34.522235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bbaf8fd3-05aa-4b53-8849-0631ad8ee3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:34.571786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:34.669028 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf8fd3-05aa-4b53-8849-0631ad8ee3dc None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:34.781523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e619b5-0e5e-4b12-89fa-08381ff54c13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:34.782093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e619b5-0e5e-4b12-89fa-08381ff54c13 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:34.782384 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e619b5-0e5e-4b12-89fa-08381ff54c13 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:34.782667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e619b5-0e5e-4b12-89fa-08381ff54c13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:34.784374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-62e619b5-0e5e-4b12-89fa-08381ff54c13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:34.791357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:34.878757 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e619b5-0e5e-4b12-89fa-08381ff54c13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:34.880428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e619b5-0e5e-4b12-89fa-08381ff54c13 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:34.899729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 717/1431] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:35.717827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d804f7c-7822-4c41-97c0-9620ab7610ef None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:35.718422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d804f7c-7822-4c41-97c0-9620ab7610ef None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:35.718422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d804f7c-7822-4c41-97c0-9620ab7610ef None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:35.719805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:35.725028 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7f2a7-b5c6-49e9-b483-e22da9348b93 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:35.726779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7f2a7-b5c6-49e9-b483-e22da9348b93 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:35.727075 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7f2a7-b5c6-49e9-b483-e22da9348b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:35.805325 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b8e7f2a7-b5c6-49e9-b483-e22da9348b93 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:35.806554 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b8e7f2a7-b5c6-49e9-b483-e22da9348b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:35.875236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:35.904860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:36.057558 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d804f7c-7822-4c41-97c0-9620ab7610ef None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:36.060447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8d804f7c-7822-4c41-97c0-9620ab7610ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:36.078715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:36.206541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d804f7c-7822-4c41-97c0-9620ab7610ef None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:36.299803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:36.330008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec395fe4-ecad-4f13-922b-32eaf9dfba91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:36.330581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec395fe4-ecad-4f13-922b-32eaf9dfba91 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:36.330871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec395fe4-ecad-4f13-922b-32eaf9dfba91 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:36.331151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec395fe4-ecad-4f13-922b-32eaf9dfba91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:36.332932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ec395fe4-ecad-4f13-922b-32eaf9dfba91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:36.437519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec395fe4-ecad-4f13-922b-32eaf9dfba91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:36.439166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec395fe4-ecad-4f13-922b-32eaf9dfba91 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:36.463366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 716/1434] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:37.297471 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed1a8a-40cd-483b-9054-a4f53022a0ad None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:37.298040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed1a8a-40cd-483b-9054-a4f53022a0ad None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:37.298139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed1a8a-40cd-483b-9054-a4f53022a0ad None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:37.299560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:37.304567 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b52fcf-6c6c-4d4e-85d2-faa4cddfbb0f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:37.305045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b52fcf-6c6c-4d4e-85d2-faa4cddfbb0f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:37.305166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b52fcf-6c6c-4d4e-85d2-faa4cddfbb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:37.389628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-43b52fcf-6c6c-4d4e-85d2-faa4cddfbb0f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:37.390854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-43b52fcf-6c6c-4d4e-85d2-faa4cddfbb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:37.432585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:37.490826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:37.651321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed1a8a-40cd-483b-9054-a4f53022a0ad None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:37.653728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5bed1a8a-40cd-483b-9054-a4f53022a0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:37.801064 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed1a8a-40cd-483b-9054-a4f53022a0ad None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:37.856864 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:37.917110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51786b80-622a-445a-8d81-0c2f9a3ccdff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:37.917835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51786b80-622a-445a-8d81-0c2f9a3ccdff service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:37.918159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51786b80-622a-445a-8d81-0c2f9a3ccdff service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:37.918477 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51786b80-622a-445a-8d81-0c2f9a3ccdff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:37.920287 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-51786b80-622a-445a-8d81-0c2f9a3ccdff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:38.011074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51786b80-622a-445a-8d81-0c2f9a3ccdff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:38.013274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51786b80-622a-445a-8d81-0c2f9a3ccdff service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:38.031550 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 720/1437] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:38.850743 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2425a8e-2697-4fd0-8a9a-5d679f53315e None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:38.851323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2425a8e-2697-4fd0-8a9a-5d679f53315e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:38.851323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2425a8e-2697-4fd0-8a9a-5d679f53315e None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:38.852724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:38.858379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2672d2-a9fe-4045-9a9d-63c812e9455d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:38.858814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2672d2-a9fe-4045-9a9d-63c812e9455d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:38.858933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2672d2-a9fe-4045-9a9d-63c812e9455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:38.934111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-7c2672d2-a9fe-4045-9a9d-63c812e9455d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:38.935321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7c2672d2-a9fe-4045-9a9d-63c812e9455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:38.937354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:39.034048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:39.183008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2425a8e-2697-4fd0-8a9a-5d679f53315e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:39.184619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e2425a8e-2697-4fd0-8a9a-5d679f53315e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:39.330220 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2425a8e-2697-4fd0-8a9a-5d679f53315e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:39.356322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:39.442667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8f17c-de9f-4740-a804-872a6f8a0c3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:39.443230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8f17c-de9f-4740-a804-872a6f8a0c3d service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:39.443543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8f17c-de9f-4740-a804-872a6f8a0c3d service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:39.443819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8f17c-de9f-4740-a804-872a6f8a0c3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:39.445487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fee8f17c-de9f-4740-a804-872a6f8a0c3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:39.537609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8f17c-de9f-4740-a804-872a6f8a0c3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:39.539239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8f17c-de9f-4740-a804-872a6f8a0c3d service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:39.558053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 719/1440] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:41.518172 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fcb0f-0de6-4773-b36f-7e229ebfda3c None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:41.518701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fcb0f-0de6-4773-b36f-7e229ebfda3c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:41.518701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fcb0f-0de6-4773-b36f-7e229ebfda3c None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:41.520054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:41.525360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4bbb8-3291-4944-9ab4-0f78520b92d0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:41.525843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4bbb8-3291-4944-9ab4-0f78520b92d0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:41.525950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4bbb8-3291-4944-9ab4-0f78520b92d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:41.567472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:41.593789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-26c4bbb8-3291-4944-9ab4-0f78520b92d0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:41.595022 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-26c4bbb8-3291-4944-9ab4-0f78520b92d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:41.694428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:41.733670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fcb0f-0de6-4773-b36f-7e229ebfda3c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:41.735197 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-125fcb0f-0de6-4773-b36f-7e229ebfda3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:41.840673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fcb0f-0de6-4773-b36f-7e229ebfda3c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:41.875570 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:41.924546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b08e21-81f1-4a79-806b-c68c6e2dcd9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:41.925252 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b08e21-81f1-4a79-806b-c68c6e2dcd9c service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:41.925503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b08e21-81f1-4a79-806b-c68c6e2dcd9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:41.925802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b08e21-81f1-4a79-806b-c68c6e2dcd9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:41.927512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b3b08e21-81f1-4a79-806b-c68c6e2dcd9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:42.040353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b08e21-81f1-4a79-806b-c68c6e2dcd9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:42.041800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b08e21-81f1-4a79-806b-c68c6e2dcd9c service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:42.062292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 723/1443] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:44.263320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0443c56-28ba-4d9b-b8c8-992e41b093c1 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:44.263918 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0443c56-28ba-4d9b-b8c8-992e41b093c1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:44.263918 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0443c56-28ba-4d9b-b8c8-992e41b093c1 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:44.265270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:44.270726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38695e66-1096-4f99-a2cf-eea98a50b056 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:44.271197 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38695e66-1096-4f99-a2cf-eea98a50b056 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:44.271399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38695e66-1096-4f99-a2cf-eea98a50b056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:44.338068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-38695e66-1096-4f99-a2cf-eea98a50b056 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:44.339940 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-38695e66-1096-4f99-a2cf-eea98a50b056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:44.353382 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:44.447163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:44.488737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0443c56-28ba-4d9b-b8c8-992e41b093c1 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:44.490295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f0443c56-28ba-4d9b-b8c8-992e41b093c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:44.610800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0443c56-28ba-4d9b-b8c8-992e41b093c1 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:44.624143 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:44.689320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9737f2-e7a5-477c-87da-eb0b826765d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:44.689969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9737f2-e7a5-477c-87da-eb0b826765d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:44.690251 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9737f2-e7a5-477c-87da-eb0b826765d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:44.690526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9737f2-e7a5-477c-87da-eb0b826765d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:44.692191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8d9737f2-e7a5-477c-87da-eb0b826765d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:44.804500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9737f2-e7a5-477c-87da-eb0b826765d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:44.805879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9737f2-e7a5-477c-87da-eb0b826765d3 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:44.835617 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 722/1446] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:47.282388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50470bbc-85a8-4fb8-8e13-30361ee5f197 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:47.282921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50470bbc-85a8-4fb8-8e13-30361ee5f197 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:47.282921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50470bbc-85a8-4fb8-8e13-30361ee5f197 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:47.284356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:47.289807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb8dc3-7cc0-4806-9dc3-12072d46aa5b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:47.290230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb8dc3-7cc0-4806-9dc3-12072d46aa5b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:47.290338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb8dc3-7cc0-4806-9dc3-12072d46aa5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:47.338503 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:47.359981 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-21bb8dc3-7cc0-4806-9dc3-12072d46aa5b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:47.361213 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-21bb8dc3-7cc0-4806-9dc3-12072d46aa5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:47.460353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:47.500893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50470bbc-85a8-4fb8-8e13-30361ee5f197 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:47.502855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-50470bbc-85a8-4fb8-8e13-30361ee5f197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:47.607464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50470bbc-85a8-4fb8-8e13-30361ee5f197 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:47.650582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:47.688285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72706b34-6a5a-486d-bcbb-6620739addf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:47.688941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72706b34-6a5a-486d-bcbb-6620739addf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:47.689228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72706b34-6a5a-486d-bcbb-6620739addf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:47.689509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72706b34-6a5a-486d-bcbb-6620739addf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:47.691151 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-72706b34-6a5a-486d-bcbb-6620739addf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:47.800383 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72706b34-6a5a-486d-bcbb-6620739addf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:47.801729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72706b34-6a5a-486d-bcbb-6620739addf3 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:47.820977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 726/1449] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:50.007181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274acd2-aca2-468e-9ea0-02c1be36b139 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:50.007769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274acd2-aca2-468e-9ea0-02c1be36b139 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:50.007769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274acd2-aca2-468e-9ea0-02c1be36b139 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:50.009122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:50.014509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942de7f6-50b9-42d9-82ee-7cfab389413d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:50.014955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942de7f6-50b9-42d9-82ee-7cfab389413d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:50.015066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942de7f6-50b9-42d9-82ee-7cfab389413d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:50.083407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-942de7f6-50b9-42d9-82ee-7cfab389413d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:50.084664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-942de7f6-50b9-42d9-82ee-7cfab389413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:50.107137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:50.185822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:50.224311 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274acd2-aca2-468e-9ea0-02c1be36b139 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:50.225871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3274acd2-aca2-468e-9ea0-02c1be36b139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:50.331005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274acd2-aca2-468e-9ea0-02c1be36b139 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:50.352681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:50.523073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f952be3-41ea-48d3-92fb-7f28d41cca42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:50.523755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f952be3-41ea-48d3-92fb-7f28d41cca42 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:50.524041 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f952be3-41ea-48d3-92fb-7f28d41cca42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:50.524319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f952be3-41ea-48d3-92fb-7f28d41cca42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:50.525961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3f952be3-41ea-48d3-92fb-7f28d41cca42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:50.634071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f952be3-41ea-48d3-92fb-7f28d41cca42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:50.635457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f952be3-41ea-48d3-92fb-7f28d41cca42 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:50.653685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 725/1452] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:52.915524 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292849-1313-43ff-8002-0831f81c3c28 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:52.916225 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292849-1313-43ff-8002-0831f81c3c28 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:52.918160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65292849-1313-43ff-8002-0831f81c3c28 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:52.918393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:00:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:52.923991 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c19d3-1edd-4ebf-aa56-c42cb59dbe63 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:52.924484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c19d3-1edd-4ebf-aa56-c42cb59dbe63 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:52.924594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c19d3-1edd-4ebf-aa56-c42cb59dbe63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:52.991434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:52.998397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-cc6c19d3-1edd-4ebf-aa56-c42cb59dbe63 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:52.999997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cc6c19d3-1edd-4ebf-aa56-c42cb59dbe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:53.102777 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:53.142941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65292849-1313-43ff-8002-0831f81c3c28 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:53.144529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-65292849-1313-43ff-8002-0831f81c3c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:53.253540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65292849-1313-43ff-8002-0831f81c3c28 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:53.302447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:53.329740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2e329-8df2-46db-bcc4-ec025f6db0fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:53.330398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2e329-8df2-46db-bcc4-ec025f6db0fa service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:53.330682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2e329-8df2-46db-bcc4-ec025f6db0fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:53.330958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2e329-8df2-46db-bcc4-ec025f6db0fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:53.332636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-01e2e329-8df2-46db-bcc4-ec025f6db0fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:53.443163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2e329-8df2-46db-bcc4-ec025f6db0fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:53.444642 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2e329-8df2-46db-bcc4-ec025f6db0fa service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:53.463953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 729/1455] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:55.589698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e542473-5f3d-485d-be0b-73aa9af9c544 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:55.590214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e542473-5f3d-485d-be0b-73aa9af9c544 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:55.590214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e542473-5f3d-485d-be0b-73aa9af9c544 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:55.591597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:55.596980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf843ca-ba2d-40ee-8b4f-8ecc50cf66c7 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:55.597424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf843ca-ba2d-40ee-8b4f-8ecc50cf66c7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:55.597550 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf843ca-ba2d-40ee-8b4f-8ecc50cf66c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:55.664096 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-adf843ca-ba2d-40ee-8b4f-8ecc50cf66c7 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:55.665332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-adf843ca-ba2d-40ee-8b4f-8ecc50cf66c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:55.695935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:55.764088 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:00:55.801927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e542473-5f3d-485d-be0b-73aa9af9c544 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:55.803514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1e542473-5f3d-485d-be0b-73aa9af9c544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:55.908436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e542473-5f3d-485d-be0b-73aa9af9c544 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:55.937838 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:55.983983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86072a-193f-449b-aa92-7a15f82d25ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:55.984637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86072a-193f-449b-aa92-7a15f82d25ca service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:55.991182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86072a-193f-449b-aa92-7a15f82d25ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:55.992475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86072a-193f-449b-aa92-7a15f82d25ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:55.994198 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6c86072a-193f-449b-aa92-7a15f82d25ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:56.108305 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86072a-193f-449b-aa92-7a15f82d25ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:56.109777 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86072a-193f-449b-aa92-7a15f82d25ca service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:56.128625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 728/1458] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:58.228607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109c0dd-7fb5-4750-8dff-e0e62a2974aa None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:58.229158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109c0dd-7fb5-4750-8dff-e0e62a2974aa None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:58.229158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109c0dd-7fb5-4750-8dff-e0e62a2974aa None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:58.230555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:00:58.236185 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4817e-3fa3-4ebf-9a8f-85093cd488b8 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:58.236629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4817e-3fa3-4ebf-9a8f-85093cd488b8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:58.236749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4817e-3fa3-4ebf-9a8f-85093cd488b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:58.300543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e4d4817e-3fa3-4ebf-9a8f-85093cd488b8 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:58.301771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e4d4817e-3fa3-4ebf-9a8f-85093cd488b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:58.310643 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:58.398496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:58.435891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109c0dd-7fb5-4750-8dff-e0e62a2974aa None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:58.437441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a109c0dd-7fb5-4750-8dff-e0e62a2974aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:58.543953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109c0dd-7fb5-4750-8dff-e0e62a2974aa None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:58.600251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:58.619494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4204f-fc20-4b8d-b47d-b948fda8f1f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:58.620786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4204f-fc20-4b8d-b47d-b948fda8f1f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:58.621078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4204f-fc20-4b8d-b47d-b948fda8f1f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:58.621359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4204f-fc20-4b8d-b47d-b948fda8f1f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:58.623018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-efe4204f-fc20-4b8d-b47d-b948fda8f1f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:58.737971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4204f-fc20-4b8d-b47d-b948fda8f1f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:58.739440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4204f-fc20-4b8d-b47d-b948fda8f1f4 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:58.758632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 732/1461] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:00.867408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b48f23-77b4-410f-b349-bb093f2c5f90 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:00.867953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b48f23-77b4-410f-b349-bb093f2c5f90 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:00.868075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b48f23-77b4-410f-b349-bb093f2c5f90 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:00.869357 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:00.874673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265fe5c-3f85-46f8-b95d-d24bc70fc4a5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:00.875108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265fe5c-3f85-46f8-b95d-d24bc70fc4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:00.875240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265fe5c-3f85-46f8-b95d-d24bc70fc4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:00.942240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0265fe5c-3f85-46f8-b95d-d24bc70fc4a5 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:00.943498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0265fe5c-3f85-46f8-b95d-d24bc70fc4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:00.993521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:01.056969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:01.095029 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b48f23-77b4-410f-b349-bb093f2c5f90 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:01.096599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-51b48f23-77b4-410f-b349-bb093f2c5f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:01.200383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b48f23-77b4-410f-b349-bb093f2c5f90 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:01.238595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:01.279995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32998069-516a-428e-abc8-ece0546f3a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:01.280648 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32998069-516a-428e-abc8-ece0546f3a4d service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:01.280930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32998069-516a-428e-abc8-ece0546f3a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:01.281204 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32998069-516a-428e-abc8-ece0546f3a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:01.282882 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-32998069-516a-428e-abc8-ece0546f3a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:01.403774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32998069-516a-428e-abc8-ece0546f3a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:01.405220 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32998069-516a-428e-abc8-ece0546f3a4d service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:01.424177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 731/1464] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:01:03.483529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec844f7b-f8b5-446c-b72d-18dfef3884cb None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:03.484148 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec844f7b-f8b5-446c-b72d-18dfef3884cb None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:03.484148 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec844f7b-f8b5-446c-b72d-18dfef3884cb None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:03.485429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:03.490927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af43a66-4068-4b52-8f8b-6c0cd2250705 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:03.491360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af43a66-4068-4b52-8f8b-6c0cd2250705 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:03.491528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af43a66-4068-4b52-8f8b-6c0cd2250705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:03.555703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5af43a66-4068-4b52-8f8b-6c0cd2250705 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:03.556974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5af43a66-4068-4b52-8f8b-6c0cd2250705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:03.574951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:03.652279 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:03.688900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec844f7b-f8b5-446c-b72d-18dfef3884cb None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:03.690433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ec844f7b-f8b5-446c-b72d-18dfef3884cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:03.793199 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec844f7b-f8b5-446c-b72d-18dfef3884cb None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:03.858734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:03.865963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafb2e0-78ca-423f-951e-ea6392ccba05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:03.866621 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafb2e0-78ca-423f-951e-ea6392ccba05 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:03.866910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafb2e0-78ca-423f-951e-ea6392ccba05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:03.867193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafb2e0-78ca-423f-951e-ea6392ccba05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:03.868875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3fafb2e0-78ca-423f-951e-ea6392ccba05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:03.979718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafb2e0-78ca-423f-951e-ea6392ccba05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:03.981190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafb2e0-78ca-423f-951e-ea6392ccba05 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:04.000687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 735/1467] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:06.065474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060631b5-70ab-44b2-885f-f844d8092518 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:06.066058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060631b5-70ab-44b2-885f-f844d8092518 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:06.066058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060631b5-70ab-44b2-885f-f844d8092518 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:06.067409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:06.072608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366014c4-6d9e-4494-a169-4247191ad58d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:06.073040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366014c4-6d9e-4494-a169-4247191ad58d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:06.073210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366014c4-6d9e-4494-a169-4247191ad58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:06.137814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-366014c4-6d9e-4494-a169-4247191ad58d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:06.139023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-366014c4-6d9e-4494-a169-4247191ad58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:06.187038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:06.240005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:06.276946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060631b5-70ab-44b2-885f-f844d8092518 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:06.278459 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-060631b5-70ab-44b2-885f-f844d8092518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:06.388093 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060631b5-70ab-44b2-885f-f844d8092518 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:06.435086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:06.460980 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b86c9-6a89-4d11-a64e-0ff555953f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:06.461625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b86c9-6a89-4d11-a64e-0ff555953f64 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:06.461904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b86c9-6a89-4d11-a64e-0ff555953f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:06.462179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b86c9-6a89-4d11-a64e-0ff555953f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:06.463819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3a1b86c9-6a89-4d11-a64e-0ff555953f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:06.570181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b86c9-6a89-4d11-a64e-0ff555953f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:06.571656 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b86c9-6a89-4d11-a64e-0ff555953f64 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:06.590228 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 734/1470] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:01:16.238133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=22 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:16.588622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=66 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:26.335321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0297ae-48d7-461e-ad16-947d355c8c9a None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:26.335907 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0297ae-48d7-461e-ad16-947d355c8c9a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:26.336021 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0297ae-48d7-461e-ad16-947d355c8c9a None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:26.337338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 737/1471] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 11:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:26.342671 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7647e4-84c7-426f-a575-329a8888ccb8 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:26.343091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7647e4-84c7-426f-a575-329a8888ccb8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:26.343225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7647e4-84c7-426f-a575-329a8888ccb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:26.406462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-1c7647e4-84c7-426f-a575-329a8888ccb8 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:26.407721 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1c7647e4-84c7-426f-a575-329a8888ccb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:26.503827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 735/1472] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 11:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:01:26.591599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c6585-1442-49c1-a122-9cf54d60e78c None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:26.591896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c6585-1442-49c1-a122-9cf54d60e78c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:26.592188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c6585-1442-49c1-a122-9cf54d60e78c None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:26.593587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 738/1473] 10.222.0.75 () {54 vars in 962 bytes} [Thu Jul 14 11:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:26.598402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2642610-d305-445a-8385-b697aa25146d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:26.598695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2642610-d305-445a-8385-b697aa25146d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:26.598979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2642610-d305-445a-8385-b697aa25146d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:26.673057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d2642610-d305-445a-8385-b697aa25146d None None] MFA Rules not processed for user `d1ad1ffe4d99420cb568d6154f779838`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:26.674269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d2642610-d305-445a-8385-b697aa25146d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:26.694188 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:26.749412 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 736/1474] 10.222.0.75 () {58 vars in 1059 bytes} [Thu Jul 14 11:01:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:26.756961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c6585-1442-49c1-a122-9cf54d60e78c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:26.758408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-407c6585-1442-49c1-a122-9cf54d60e78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:26.861470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c6585-1442-49c1-a122-9cf54d60e78c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:26.946260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b346c2-8f5c-43bf-9d10-b443b975670e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1ad1ffe4d99420cb568d6154f779838', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:26.946922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b346c2-8f5c-43bf-9d10-b443b975670e service glance] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:26.947209 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b346c2-8f5c-43bf-9d10-b443b975670e service glance] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:26.947521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b346c2-8f5c-43bf-9d10-b443b975670e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:26.949192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-87b346c2-8f5c-43bf-9d10-b443b975670e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:27.005158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:27.060357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b346c2-8f5c-43bf-9d10-b443b975670e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:27.061700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b346c2-8f5c-43bf-9d10-b443b975670e service glance] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:27.080309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 739/1475] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:29.211728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057c364-e69b-42c7-87f1-2d7222fd89a5 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:29.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057c364-e69b-42c7-87f1-2d7222fd89a5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:29.212971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057c364-e69b-42c7-87f1-2d7222fd89a5 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:29.213691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 737/1476] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 11:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:29.218976 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7a6fb-904a-4b65-98a7-1691c043b3f3 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:29.219430 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7a6fb-904a-4b65-98a7-1691c043b3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:29.219573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7a6fb-904a-4b65-98a7-1691c043b3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:29.292307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d2c7a6fb-904a-4b65-98a7-1691c043b3f3 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:29.293536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d2c7a6fb-904a-4b65-98a7-1691c043b3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:29.400252 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 740/1477] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 11:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:29.424128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057c364-e69b-42c7-87f1-2d7222fd89a5 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:29.425714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3057c364-e69b-42c7-87f1-2d7222fd89a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:29.561810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057c364-e69b-42c7-87f1-2d7222fd89a5 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:29.611103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:29.658822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35061e7e-ced5-40be-9634-36b9c191acbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:29.659545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35061e7e-ced5-40be-9634-36b9c191acbf admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:29.659859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35061e7e-ced5-40be-9634-36b9c191acbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:29.660136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35061e7e-ced5-40be-9634-36b9c191acbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:29.660868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35061e7e-ced5-40be-9634-36b9c191acbf admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:29.662633 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-35061e7e-ced5-40be-9634-36b9c191acbf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:01:29.662633 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 14 11:01:29.662633 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: Jul 14 11:01:29.663813 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35061e7e-ced5-40be-9634-36b9c191acbf admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:29.685949 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 738/1478] 10.222.0.75 () {56 vars in 1108 bytes} [Thu Jul 14 11:01:29 2022] GET /identity/v3/projects => generated 2553 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:01:31.820388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0cf69-2abb-4fa8-8936-7f153a906d04 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:31.820967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0cf69-2abb-4fa8-8936-7f153a906d04 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:31.820967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0cf69-2abb-4fa8-8936-7f153a906d04 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:31.822874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 741/1479] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 11:01:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:31.828330 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08a069-3de4-4f8e-94e8-988818823b2a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:31.828820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08a069-3de4-4f8e-94e8-988818823b2a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:31.828936 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08a069-3de4-4f8e-94e8-988818823b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:31.904706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ff08a069-3de4-4f8e-94e8-988818823b2a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:31.905934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ff08a069-3de4-4f8e-94e8-988818823b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:32.004583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 739/1480] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 11:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:32.048838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0cf69-2abb-4fa8-8936-7f153a906d04 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:32.050383 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f9b0cf69-2abb-4fa8-8936-7f153a906d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:32.061410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:32.202435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0cf69-2abb-4fa8-8936-7f153a906d04 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:32.282716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:32.316109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01937a6-eebe-4194-8262-6013408bd949 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:32.316761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01937a6-eebe-4194-8262-6013408bd949 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:32.317043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01937a6-eebe-4194-8262-6013408bd949 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:32.317317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01937a6-eebe-4194-8262-6013408bd949 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:32.318938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c01937a6-eebe-4194-8262-6013408bd949 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:32.406477 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01937a6-eebe-4194-8262-6013408bd949 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:32.408104 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01937a6-eebe-4194-8262-6013408bd949 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:32.426377 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 742/1481] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:34.673106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277fdb8-692e-431d-a00b-cb560158e971 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:34.673755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277fdb8-692e-431d-a00b-cb560158e971 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:34.673755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277fdb8-692e-431d-a00b-cb560158e971 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:34.675033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 740/1482] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 11:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:34.680450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a283c0-602a-4b88-b114-668cd2c7c37c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:34.680918 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a283c0-602a-4b88-b114-668cd2c7c37c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:34.681024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a283c0-602a-4b88-b114-668cd2c7c37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:34.745914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e2a283c0-602a-4b88-b114-668cd2c7c37c None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:34.747128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e2a283c0-602a-4b88-b114-668cd2c7c37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:34.793312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:34.844297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 743/1483] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 11:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:34.880793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277fdb8-692e-431d-a00b-cb560158e971 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:34.882343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c277fdb8-692e-431d-a00b-cb560158e971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:34.907007 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:34.986998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277fdb8-692e-431d-a00b-cb560158e971 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:35.071402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a3727-9bfc-4855-b859-0cb2a9e5cb5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:35.072629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a3727-9bfc-4855-b859-0cb2a9e5cb5a service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:35.072911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a3727-9bfc-4855-b859-0cb2a9e5cb5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:35.073185 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a3727-9bfc-4855-b859-0cb2a9e5cb5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:35.074839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-016a3727-9bfc-4855-b859-0cb2a9e5cb5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:35.148165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:35.184365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a3727-9bfc-4855-b859-0cb2a9e5cb5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:35.185834 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a3727-9bfc-4855-b859-0cb2a9e5cb5a service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:35.205491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 741/1484] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:37.329263 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682813-d4af-4de8-a54e-99bf8aead939 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.329804 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682813-d4af-4de8-a54e-99bf8aead939 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.329922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682813-d4af-4de8-a54e-99bf8aead939 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.331218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 744/1485] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 11:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:37.336671 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff66d0a-887c-493d-a178-f1146e787de4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.337166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff66d0a-887c-493d-a178-f1146e787de4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.337273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff66d0a-887c-493d-a178-f1146e787de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.401970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-eff66d0a-887c-493d-a178-f1146e787de4 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:37.403207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-eff66d0a-887c-493d-a178-f1146e787de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:37.500191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 742/1486] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 11:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:37.544379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40682813-d4af-4de8-a54e-99bf8aead939 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.545926 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-40682813-d4af-4de8-a54e-99bf8aead939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:37.656453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40682813-d4af-4de8-a54e-99bf8aead939 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.730978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:37.733033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75159c49-9585-4a37-870f-5c3bf4aea535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.733908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75159c49-9585-4a37-870f-5c3bf4aea535 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.734364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75159c49-9585-4a37-870f-5c3bf4aea535 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.734808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75159c49-9585-4a37-870f-5c3bf4aea535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.736792 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-75159c49-9585-4a37-870f-5c3bf4aea535 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:37.859153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75159c49-9585-4a37-870f-5c3bf4aea535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.860684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75159c49-9585-4a37-870f-5c3bf4aea535 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.882429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 745/1487] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:40.158934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9716971-6db7-443f-ade3-11b6f301e542 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:40.159808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9716971-6db7-443f-ade3-11b6f301e542 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:40.159808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9716971-6db7-443f-ade3-11b6f301e542 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:40.161040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 743/1488] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 11:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:40.167464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cf1e6-9e97-4fda-9e54-73a79bb9186b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:40.168307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cf1e6-9e97-4fda-9e54-73a79bb9186b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:40.168307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cf1e6-9e97-4fda-9e54-73a79bb9186b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:40.250982 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-786cf1e6-9e97-4fda-9e54-73a79bb9186b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:40.252254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-786cf1e6-9e97-4fda-9e54-73a79bb9186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:40.318603 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:40.373129 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 746/1489] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 11:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:40.415300 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9716971-6db7-443f-ade3-11b6f301e542 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:40.417033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b9716971-6db7-443f-ade3-11b6f301e542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:40.461109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:40.554407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9716971-6db7-443f-ade3-11b6f301e542 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:40.643328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f31ea-1f1f-4029-b590-2d46b2cd8630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:40.644037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f31ea-1f1f-4029-b590-2d46b2cd8630 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:40.644327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f31ea-1f1f-4029-b590-2d46b2cd8630 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:40.644609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f31ea-1f1f-4029-b590-2d46b2cd8630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:40.646352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-990f31ea-1f1f-4029-b590-2d46b2cd8630 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:40.738414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:40.776594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f31ea-1f1f-4029-b590-2d46b2cd8630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:40.778098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f31ea-1f1f-4029-b590-2d46b2cd8630 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:40.800850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 744/1490] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:01:43.195342 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a9bb1-94e3-4a71-8b96-19f1d3aa09b2 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:43.195862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a9bb1-94e3-4a71-8b96-19f1d3aa09b2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:43.195862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a9bb1-94e3-4a71-8b96-19f1d3aa09b2 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:43.198845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 747/1491] 10.222.0.75 () {54 vars in 926 bytes} [Thu Jul 14 11:01:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:01:43.205424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d976eeb-a904-418e-bcfc-9dd7cab9f01a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:43.206030 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d976eeb-a904-418e-bcfc-9dd7cab9f01a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:43.206030 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d976eeb-a904-418e-bcfc-9dd7cab9f01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:43.282582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2d976eeb-a904-418e-bcfc-9dd7cab9f01a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:43.283861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2d976eeb-a904-418e-bcfc-9dd7cab9f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:43.385364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 745/1492] 10.222.0.75 () {58 vars in 1023 bytes} [Thu Jul 14 11:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:01:43.424699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a9bb1-94e3-4a71-8b96-19f1d3aa09b2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:43.426306 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9a9a9bb1-94e3-4a71-8b96-19f1d3aa09b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:43.531961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a9bb1-94e3-4a71-8b96-19f1d3aa09b2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:43.606593 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39538e53-891c-49e5-9bd5-141c25f1119b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:43.607243 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39538e53-891c-49e5-9bd5-141c25f1119b service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:43.607562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39538e53-891c-49e5-9bd5-141c25f1119b service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:43.607847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39538e53-891c-49e5-9bd5-141c25f1119b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:43.609496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-39538e53-891c-49e5-9bd5-141c25f1119b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:43.619246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:43.718575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39538e53-891c-49e5-9bd5-141c25f1119b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:43.720052 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39538e53-891c-49e5-9bd5-141c25f1119b service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:43.738519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 748/1493] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:53.382945 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=82 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:53.736186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=50 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:20.520803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68954b80-da8c-4699-a574-3e40847ce1ee None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:20.521715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68954b80-da8c-4699-a574-3e40847ce1ee None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:20.521715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68954b80-da8c-4699-a574-3e40847ce1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:20.638774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-68954b80-da8c-4699-a574-3e40847ce1ee None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:20.640054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-68954b80-da8c-4699-a574-3e40847ce1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:20.746382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 746/1494] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:03:20.754812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7e48d-b2e0-47ca-ab6f-756410df031c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:20.755525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7e48d-b2e0-47ca-ab6f-756410df031c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:20.755525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7e48d-b2e0-47ca-ab6f-756410df031c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:20.839639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-bae7e48d-b2e0-47ca-ab6f-756410df031c None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:20.840887 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bae7e48d-b2e0-47ca-ab6f-756410df031c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:20.995919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 749/1495] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:21.003249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68954b80-da8c-4699-a574-3e40847ce1ee None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:21.005950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-68954b80-da8c-4699-a574-3e40847ce1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:21.027898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68954b80-da8c-4699-a574-3e40847ce1ee None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:21.118209 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:21.156149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca14b62-f066-40c9-86af-1aa81e75990b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:21.156884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca14b62-f066-40c9-86af-1aa81e75990b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:21.157223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca14b62-f066-40c9-86af-1aa81e75990b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:21.157542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca14b62-f066-40c9-86af-1aa81e75990b admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:21.158421 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca14b62-f066-40c9-86af-1aa81e75990b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:03:21.158767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca14b62-f066-40c9-86af-1aa81e75990b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:21.160172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-7ca14b62-f066-40c9-86af-1aa81e75990b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:03:21.160172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 14 11:03:21.160172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: Jul 14 11:03:21.161403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca14b62-f066-40c9-86af-1aa81e75990b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:21.174996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 747/1496] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:03:21.184297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7e48d-b2e0-47ca-ab6f-756410df031c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:21.196251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7e48d-b2e0-47ca-ab6f-756410df031c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:21.272636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:21.315591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfffa3e4-b10b-4eb9-93fd-ba44044fad9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:21.316319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffa3e4-b10b-4eb9-93fd-ba44044fad9f admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:21.316655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffa3e4-b10b-4eb9-93fd-ba44044fad9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:21.316972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffa3e4-b10b-4eb9-93fd-ba44044fad9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:21.318372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfffa3e4-b10b-4eb9-93fd-ba44044fad9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:21.320331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-bfffa3e4-b10b-4eb9-93fd-ba44044fad9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:03:21.320331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 14 11:03:21.320331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: Jul 14 11:03:21.321727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfffa3e4-b10b-4eb9-93fd-ba44044fad9f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:21.391752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 750/1497] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:03:21 2022] POST /identity/v3/projects => generated 379 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:03:21.396307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca14b62-f066-40c9-86af-1aa81e75990b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:21.397889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7ca14b62-f066-40c9-86af-1aa81e75990b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:21.435698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca14b62-f066-40c9-86af-1aa81e75990b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:21.546424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dd8e8-2012-4bba-9fbb-ae53153fb2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:21.547129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dd8e8-2012-4bba-9fbb-ae53153fb2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:21.547290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dd8e8-2012-4bba-9fbb-ae53153fb2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:21.547577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dd8e8-2012-4bba-9fbb-ae53153fb2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:21.548285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8dd8e8-2012-4bba-9fbb-ae53153fb2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:21.549327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-2d8dd8e8-2012-4bba-9fbb-ae53153fb2f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:03:21.549327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 14 11:03:21.549327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: Jul 14 11:03:21.550565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8dd8e8-2012-4bba-9fbb-ae53153fb2f1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:21.602770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 748/1498] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:03:21 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:21.607141 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfffa3e4-b10b-4eb9-93fd-ba44044fad9f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:21.608894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bfffa3e4-b10b-4eb9-93fd-ba44044fad9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:21.679803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfffa3e4-b10b-4eb9-93fd-ba44044fad9f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:21.774371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:21.788891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382113fb-5e57-4e6b-a1a8-411f37a3e781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:21.789578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382113fb-5e57-4e6b-a1a8-411f37a3e781 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:21.789860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382113fb-5e57-4e6b-a1a8-411f37a3e781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:21.790131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382113fb-5e57-4e6b-a1a8-411f37a3e781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:21.790924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382113fb-5e57-4e6b-a1a8-411f37a3e781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:21.791975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-382113fb-5e57-4e6b-a1a8-411f37a3e781 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:03:21.791975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 14 11:03:21.791975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: Jul 14 11:03:21.792870 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382113fb-5e57-4e6b-a1a8-411f37a3e781 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:21.807750 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 751/1499] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:21.812467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dd8e8-2012-4bba-9fbb-ae53153fb2f1 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:21.814016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2d8dd8e8-2012-4bba-9fbb-ae53153fb2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:21.826270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dd8e8-2012-4bba-9fbb-ae53153fb2f1 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:21.852494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:21.929159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04571283-0591-4106-b2ad-0edf67767b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:21.929851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04571283-0591-4106-b2ad-0edf67767b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:21.930118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04571283-0591-4106-b2ad-0edf67767b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:21.930392 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04571283-0591-4106-b2ad-0edf67767b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:21.931507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04571283-0591-4106-b2ad-0edf67767b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:21.932473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-04571283-0591-4106-b2ad-0edf67767b4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:03:21.932473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 14 11:03:21.932473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: Jul 14 11:03:21.933392 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04571283-0591-4106-b2ad-0edf67767b4e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:21.946734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 749/1500] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:21.951341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382113fb-5e57-4e6b-a1a8-411f37a3e781 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:21.953059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-382113fb-5e57-4e6b-a1a8-411f37a3e781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:21.990273 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382113fb-5e57-4e6b-a1a8-411f37a3e781 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:22.089047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa13df5-7b27-4284-8d1d-172f035ed701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:22.089843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa13df5-7b27-4284-8d1d-172f035ed701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:22.090129 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa13df5-7b27-4284-8d1d-172f035ed701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:22.090417 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa13df5-7b27-4284-8d1d-172f035ed701 admin admin] PATH_INFO: `/v3/projects/8de7ef28120646d0b46f031b52510c97/users/da56f178c55241ca985c48a10fd34055/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:22.115189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa13df5-7b27-4284-8d1d-172f035ed701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8de7ef28120646d0b46f031b52510c97, user_id=da56f178c55241ca985c48a10fd34055, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:22.116278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-cfa13df5-7b27-4284-8d1d-172f035ed701 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:03:22.116278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 14 11:03:22.116278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: Jul 14 11:03:22.118285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa13df5-7b27-4284-8d1d-172f035ed701 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:22.144448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 752/1501] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:03:21 2022] PUT /identity/v3/projects/8de7ef28120646d0b46f031b52510c97/users/da56f178c55241ca985c48a10fd34055/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:03:22.157547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552d3e7-1a78-4d0f-8a30-80bf1bfb49bd None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:22.158070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552d3e7-1a78-4d0f-8a30-80bf1bfb49bd None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:22.158425 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552d3e7-1a78-4d0f-8a30-80bf1bfb49bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:22.222755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f552d3e7-1a78-4d0f-8a30-80bf1bfb49bd None None] MFA Rules not processed for user `da56f178c55241ca985c48a10fd34055`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:22.224614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f552d3e7-1a78-4d0f-8a30-80bf1bfb49bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:22.312094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 750/1502] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:03:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:22.325949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcd5da-54a4-49f0-93e8-510eba996208 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:22.326238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcd5da-54a4-49f0-93e8-510eba996208 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:22.326523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcd5da-54a4-49f0-93e8-510eba996208 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:22.327962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 753/1503] 10.222.0.75 () {54 vars in 963 bytes} [Thu Jul 14 11:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:03:22.333787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f12a71-a5d4-4678-ab9f-e734b52cceb6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:22.334078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f12a71-a5d4-4678-ab9f-e734b52cceb6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:22.334378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f12a71-a5d4-4678-ab9f-e734b52cceb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:22.365951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:22.413898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-93f12a71-a5d4-4678-ab9f-e734b52cceb6 None None] MFA Rules not processed for user `80230b1b73944e8ebb6fec3dbe0d5c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:22.415132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-93f12a71-a5d4-4678-ab9f-e734b52cceb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:22.492769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 751/1504] 10.222.0.75 () {58 vars in 1060 bytes} [Thu Jul 14 11:03:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:22.500600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fcd5da-54a4-49f0-93e8-510eba996208 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:22.502116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-86fcd5da-54a4-49f0-93e8-510eba996208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:22.559565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:22.625749 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fcd5da-54a4-49f0-93e8-510eba996208 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:22.717573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf7a88-f157-4b4b-b808-f2882355ea40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:22.718236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf7a88-f157-4b4b-b808-f2882355ea40 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:22.718523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf7a88-f157-4b4b-b808-f2882355ea40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:22.718804 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf7a88-f157-4b4b-b808-f2882355ea40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:22.720856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1edf7a88-f157-4b4b-b808-f2882355ea40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:22.840809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf7a88-f157-4b4b-b808-f2882355ea40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:22.842302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf7a88-f157-4b4b-b808-f2882355ea40 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:22.846418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:22.867608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 754/1505] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:22.926481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77380971-ea93-49f7-9b39-620f1de84735 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:22.927543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77380971-ea93-49f7-9b39-620f1de84735 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:22.927650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77380971-ea93-49f7-9b39-620f1de84735 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:22.928896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 752/1506] 10.222.0.75 () {52 vars in 829 bytes} [Thu Jul 14 11:03:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:03:22.933736 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf7a88-f157-4b4b-b808-f2882355ea40 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:22.935470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1edf7a88-f157-4b4b-b808-f2882355ea40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:23.060150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf7a88-f157-4b4b-b808-f2882355ea40 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:23.159256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:23.160171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:23.160456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:23.160737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] PATH_INFO: `/v3/users/da56f178c55241ca985c48a10fd34055` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:23.174863 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da56f178c55241ca985c48a10fd34055)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:23.175946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:03:23.175946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 14 11:03:23.175946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: Jul 14 11:03:23.177709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:23.189443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:23.261160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da56f178c55241ca985c48a10fd34055'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:23.261724 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da56f178c55241ca985c48a10fd34055'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:23.276146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da56f178c55241ca985c48a10fd34055'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:23.289165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da56f178c55241ca985c48a10fd34055'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:23.289165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da56f178c55241ca985c48a10fd34055'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:23.314847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 755/1507] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:03:22 2022] DELETE /identity/v3/users/da56f178c55241ca985c48a10fd34055 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:23.321650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9689b12-fbfb-4de0-a0c7-5652f213ccae None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:23.322175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9689b12-fbfb-4de0-a0c7-5652f213ccae None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:23.322691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9689b12-fbfb-4de0-a0c7-5652f213ccae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:23.406431 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e9689b12-fbfb-4de0-a0c7-5652f213ccae None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:23.408624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e9689b12-fbfb-4de0-a0c7-5652f213ccae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:23.498807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:23.514635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 753/1508] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:23.526261 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:23.527568 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:23.651787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34e7ba-99f6-4501-84c6-0964d80f6643 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:23.713494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:23.736891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b2d96-af69-4c76-ba29-b877b1a1c072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:23.737567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b2d96-af69-4c76-ba29-b877b1a1c072 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:23.738002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b2d96-af69-4c76-ba29-b877b1a1c072 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:23.738330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b2d96-af69-4c76-ba29-b877b1a1c072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:23.741000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-915b2d96-af69-4c76-ba29-b877b1a1c072 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:23.869699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b2d96-af69-4c76-ba29-b877b1a1c072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:23.871075 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b2d96-af69-4c76-ba29-b877b1a1c072 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:23.894886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 756/1509] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:24.381402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9689b12-fbfb-4de0-a0c7-5652f213ccae None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:24.383201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e9689b12-fbfb-4de0-a0c7-5652f213ccae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:24.399965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9689b12-fbfb-4de0-a0c7-5652f213ccae None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:24.526363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:24.527251 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:24.527570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:24.527845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] PATH_INFO: `/v3/projects/8de7ef28120646d0b46f031b52510c97` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:24.542886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8de7ef28120646d0b46f031b52510c97)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:24.543994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:03:24.543994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 14 11:03:24.543994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: Jul 14 11:03:24.545227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:24.615255 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8de7ef28120646d0b46f031b52510c97', 'request_id': 'req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:24.629906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8de7ef28120646d0b46f031b52510c97', 'request_id': 'req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:24.637695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8de7ef28120646d0b46f031b52510c97', 'request_id': 'req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:24.672906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] The token cache is being invalidate because project 8de7ef28120646d0b46f031b52510c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:03:24.673252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:24.673899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8482e6ca-7ae8-45e6-9383-b8af773a8e4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:24.675938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 754/1510] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:03:24 2022] DELETE /identity/v3/projects/8de7ef28120646d0b46f031b52510c97 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:33.892268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=62 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:34.670087 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:05:08.391938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db264f7f-2281-4e9c-a156-565505a3b155 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:08.392498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db264f7f-2281-4e9c-a156-565505a3b155 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:08.392498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db264f7f-2281-4e9c-a156-565505a3b155 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:08.393905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:05:08.399342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5812bea-b3f7-4352-bf5a-1ac4de732d64 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:08.399869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5812bea-b3f7-4352-bf5a-1ac4de732d64 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:08.400045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5812bea-b3f7-4352-bf5a-1ac4de732d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:08.507295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f5812bea-b3f7-4352-bf5a-1ac4de732d64 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:05:08.509059 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f5812bea-b3f7-4352-bf5a-1ac4de732d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:08.624625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:05:08.780465 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db264f7f-2281-4e9c-a156-565505a3b155 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:05:08.783423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-db264f7f-2281-4e9c-a156-565505a3b155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:08.909716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db264f7f-2281-4e9c-a156-565505a3b155 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:05:08.915352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa5f18-2fc3-403e-a41b-a281b04d9435 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:05:08.916058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa5f18-2fc3-403e-a41b-a281b04d9435 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:08.916344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa5f18-2fc3-403e-a41b-a281b04d9435 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:08.916629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa5f18-2fc3-403e-a41b-a281b04d9435 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:08.926845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa5f18-2fc3-403e-a41b-a281b04d9435 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:05:08.928588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa5f18-2fc3-403e-a41b-a281b04d9435 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:05:08.939288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 758/1513] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:05:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:05:09.832849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19311f07-bf3d-4489-85c9-4ed643c50663 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:09.833482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19311f07-bf3d-4489-85c9-4ed643c50663 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:09.833482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19311f07-bf3d-4489-85c9-4ed643c50663 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:09.834849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:05:09.840306 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e537a0-7bb0-4537-93a9-23ed3d96b944 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:09.840845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e537a0-7bb0-4537-93a9-23ed3d96b944 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:09.840845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e537a0-7bb0-4537-93a9-23ed3d96b944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:09.859798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-39e537a0-7bb0-4537-93a9-23ed3d96b944 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:05:09.861579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-39e537a0-7bb0-4537-93a9-23ed3d96b944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:09.871743 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:05:10.028464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19311f07-bf3d-4489-85c9-4ed643c50663 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:05:10.046329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19311f07-bf3d-4489-85c9-4ed643c50663 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:05:10.052870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198d7fd-2a2d-41b1-a4e6-a930974a0c6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:05:10.053563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198d7fd-2a2d-41b1-a4e6-a930974a0c6c service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:10.053858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198d7fd-2a2d-41b1-a4e6-a930974a0c6c service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:10.054133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198d7fd-2a2d-41b1-a4e6-a930974a0c6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:10.064972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198d7fd-2a2d-41b1-a4e6-a930974a0c6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:05:10.066645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198d7fd-2a2d-41b1-a4e6-a930974a0c6c service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:05:10.077905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 757/1516] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:05:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:05:10.236487 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6891d03-c05d-456c-8ee9-269e400f2465 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:10.236943 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6891d03-c05d-456c-8ee9-269e400f2465 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:10.237082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6891d03-c05d-456c-8ee9-269e400f2465 None None] PATH_INFO: `/v3` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:10.238632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 760/1517] 10.222.0.26 () {54 vars in 925 bytes} [Thu Jul 14 11:05:10 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:05:10.245552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6d4f1-3e08-4016-95f8-7a3c6ab22692 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:10.245830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6d4f1-3e08-4016-95f8-7a3c6ab22692 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:10.246107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6d4f1-3e08-4016-95f8-7a3c6ab22692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:10.316372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2ca6d4f1-3e08-4016-95f8-7a3c6ab22692 None None] MFA Rules not processed for user `0fc1f2abce4845219741f4f48bf32d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:05:10.317605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2ca6d4f1-3e08-4016-95f8-7a3c6ab22692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:10.336214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:05:10.414461 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 758/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Jul 14 11:05:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:05:19.582728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9495ae-e3e9-4fc9-b2d5-d4b2bb929ee2 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:19.583262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9495ae-e3e9-4fc9-b2d5-d4b2bb929ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:19.583262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9495ae-e3e9-4fc9-b2d5-d4b2bb929ee2 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:19.584614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 761/1519] 10.222.0.75 () {54 vars in 960 bytes} [Thu Jul 14 11:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:05:19.589744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd61cf-b8e5-4a70-9320-96eda4cf30f3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:19.590171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd61cf-b8e5-4a70-9320-96eda4cf30f3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:19.590288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd61cf-b8e5-4a70-9320-96eda4cf30f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:19.607801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-15dd61cf-b8e5-4a70-9320-96eda4cf30f3 None None] MFA Rules not processed for user `0fc1f2abce4845219741f4f48bf32d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:05:19.608991 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-15dd61cf-b8e5-4a70-9320-96eda4cf30f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:19.617910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 759/1520] 10.222.0.75 () {58 vars in 1057 bytes} [Thu Jul 14 11:05:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:05:19.625290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9495ae-e3e9-4fc9-b2d5-d4b2bb929ee2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:05:19.639848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9495ae-e3e9-4fc9-b2d5-d4b2bb929ee2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:05:19.644983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6e0b9-6e99-4ebf-9caf-b1769ffe6a32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:05:19.645629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6e0b9-6e99-4ebf-9caf-b1769ffe6a32 service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:19.645917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6e0b9-6e99-4ebf-9caf-b1769ffe6a32 service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:19.646199 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6e0b9-6e99-4ebf-9caf-b1769ffe6a32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:19.654707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6e0b9-6e99-4ebf-9caf-b1769ffe6a32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:05:19.656328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6e0b9-6e99-4ebf-9caf-b1769ffe6a32 service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:05:19.666290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 762/1521] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:05:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:05:21.337178 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6d27a-9e0c-4cca-afa6-8638d619edd7 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:21.337774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6d27a-9e0c-4cca-afa6-8638d619edd7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:21.337774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6d27a-9e0c-4cca-afa6-8638d619edd7 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:21.339112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:05:21.344015 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d9e48-0275-48e4-8ccb-246e5587271f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:21.344478 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d9e48-0275-48e4-8ccb-246e5587271f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:21.344579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d9e48-0275-48e4-8ccb-246e5587271f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:21.362956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-960d9e48-0275-48e4-8ccb-246e5587271f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:05:21.364197 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-960d9e48-0275-48e4-8ccb-246e5587271f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:21.372058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:05:21.524582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6d27a-9e0c-4cca-afa6-8638d619edd7 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:05:21.541006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6d27a-9e0c-4cca-afa6-8638d619edd7 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:05:21.550941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4aa051-2425-4768-9f3c-0a6c5db1df27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:05:21.551723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4aa051-2425-4768-9f3c-0a6c5db1df27 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:21.552004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4aa051-2425-4768-9f3c-0a6c5db1df27 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:21.552357 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4aa051-2425-4768-9f3c-0a6c5db1df27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:21.562637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4aa051-2425-4768-9f3c-0a6c5db1df27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:05:21.564509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4aa051-2425-4768-9f3c-0a6c5db1df27 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:05:21.576096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 761/1524] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:05:22.414959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f51cea-eca2-414e-850b-8dfb0c71f603 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:22.415481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f51cea-eca2-414e-850b-8dfb0c71f603 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:22.415578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f51cea-eca2-414e-850b-8dfb0c71f603 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:22.416970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 11:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:05:22.421906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69776eb0-b577-48e0-b010-4c771ccb4a36 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:22.422374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69776eb0-b577-48e0-b010-4c771ccb4a36 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:22.422496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69776eb0-b577-48e0-b010-4c771ccb4a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:22.441031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-69776eb0-b577-48e0-b010-4c771ccb4a36 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:05:22.442239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-69776eb0-b577-48e0-b010-4c771ccb4a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:22.450279 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 11:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:05:22.602601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f51cea-eca2-414e-850b-8dfb0c71f603 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:05:22.617396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f51cea-eca2-414e-850b-8dfb0c71f603 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:05:22.622859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4d609-a5c5-4e8f-ac7a-4f2b77130a42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:05:22.623532 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4d609-a5c5-4e8f-ac7a-4f2b77130a42 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:22.623823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4d609-a5c5-4e8f-ac7a-4f2b77130a42 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:22.624102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4d609-a5c5-4e8f-ac7a-4f2b77130a42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:22.633582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4d609-a5c5-4e8f-ac7a-4f2b77130a42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:05:22.635184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4d609-a5c5-4e8f-ac7a-4f2b77130a42 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:05:22.644850 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 765/1527] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:05:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:05:26.754898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55445bf-e684-4d77-bfc2-972ce903918a None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:26.755525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55445bf-e684-4d77-bfc2-972ce903918a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:26.755525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55445bf-e684-4d77-bfc2-972ce903918a None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:26.756891 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:05:26.762085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799d2c5-dc1f-4b86-a211-c55f45d60274 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:26.762548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799d2c5-dc1f-4b86-a211-c55f45d60274 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:26.762649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799d2c5-dc1f-4b86-a211-c55f45d60274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:26.780540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1799d2c5-dc1f-4b86-a211-c55f45d60274 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:05:26.782269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1799d2c5-dc1f-4b86-a211-c55f45d60274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:26.790272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:05:26.864636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55445bf-e684-4d77-bfc2-972ce903918a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:05:26.866370 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e55445bf-e684-4d77-bfc2-972ce903918a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:26.944108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55445bf-e684-4d77-bfc2-972ce903918a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:05:26.947644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cdc6e-e274-4819-b590-dee7c9e48f59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:05:26.948286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cdc6e-e274-4819-b590-dee7c9e48f59 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:26.948567 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cdc6e-e274-4819-b590-dee7c9e48f59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:26.948840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cdc6e-e274-4819-b590-dee7c9e48f59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:26.957576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89cdc6e-e274-4819-b590-dee7c9e48f59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:05:26.958997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89cdc6e-e274-4819-b590-dee7c9e48f59 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:05:26.968441 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 764/1530] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:05:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:05:29.005312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c3ffb-eb37-413d-93b1-a552dd72996f None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:29.005905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c3ffb-eb37-413d-93b1-a552dd72996f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:29.005905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c3ffb-eb37-413d-93b1-a552dd72996f None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:29.007256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:05:29.012688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021713f-6560-4aa5-9292-006c7e04eeb4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:29.013138 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021713f-6560-4aa5-9292-006c7e04eeb4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:29.013261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021713f-6560-4aa5-9292-006c7e04eeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:29.030921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b021713f-6560-4aa5-9292-006c7e04eeb4 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:05:29.032685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b021713f-6560-4aa5-9292-006c7e04eeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:29.040471 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:05:29.075692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c3ffb-eb37-413d-93b1-a552dd72996f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:05:29.086398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c3ffb-eb37-413d-93b1-a552dd72996f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:05:29.089857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c923c51-6afd-48b5-8120-9eb51935759c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:05:29.090497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c923c51-6afd-48b5-8120-9eb51935759c service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:29.090783 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c923c51-6afd-48b5-8120-9eb51935759c service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:29.091058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c923c51-6afd-48b5-8120-9eb51935759c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:29.100549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c923c51-6afd-48b5-8120-9eb51935759c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:05:29.101952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c923c51-6afd-48b5-8120-9eb51935759c service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:05:29.111958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 768/1533] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:05:32.628653 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909c611-9270-4113-b71d-9083d6ceac47 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:32.629280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909c611-9270-4113-b71d-9083d6ceac47 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:32.629280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909c611-9270-4113-b71d-9083d6ceac47 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:32.630498 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:05:32.635798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b9562-e77b-4d15-9e6a-e13bf91d5560 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:32.636275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b9562-e77b-4d15-9e6a-e13bf91d5560 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:32.636394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b9562-e77b-4d15-9e6a-e13bf91d5560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:32.660123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-365b9562-e77b-4d15-9e6a-e13bf91d5560 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:05:32.661355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-365b9562-e77b-4d15-9e6a-e13bf91d5560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:32.669427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:05:32.977615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909c611-9270-4113-b71d-9083d6ceac47 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:05:32.979339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8909c611-9270-4113-b71d-9083d6ceac47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:33.043965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:05:33.058399 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909c611-9270-4113-b71d-9083d6ceac47 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:05:33.062228 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1e97f-409c-4b0d-8a37-cc85cb66a0d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:05:33.063494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1e97f-409c-4b0d-8a37-cc85cb66a0d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:33.063785 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1e97f-409c-4b0d-8a37-cc85cb66a0d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:33.064064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1e97f-409c-4b0d-8a37-cc85cb66a0d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:33.073349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1e97f-409c-4b0d-8a37-cc85cb66a0d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:05:33.074797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1e97f-409c-4b0d-8a37-cc85cb66a0d0 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:05:33.084419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 767/1536] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:05:35.150801 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec1698-9916-47f9-8c6e-0a8f5bef46b0 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:35.151341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec1698-9916-47f9-8c6e-0a8f5bef46b0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:35.151496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec1698-9916-47f9-8c6e-0a8f5bef46b0 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:35.152781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 11:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:05:35.158369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4ac39-0b0b-4b2f-85d7-d23050db6e4d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:35.158817 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4ac39-0b0b-4b2f-85d7-d23050db6e4d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:35.158914 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4ac39-0b0b-4b2f-85d7-d23050db6e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:35.177602 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-dda4ac39-0b0b-4b2f-85d7-d23050db6e4d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:05:35.178811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-dda4ac39-0b0b-4b2f-85d7-d23050db6e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:35.187086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 11:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:05:35.488365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec1698-9916-47f9-8c6e-0a8f5bef46b0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:05:35.490082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-70ec1698-9916-47f9-8c6e-0a8f5bef46b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:35.498043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=84 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:05:35.503968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec1698-9916-47f9-8c6e-0a8f5bef46b0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:05:35.507608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa636a-67d5-4cda-8bfb-38cf262e2df9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:05:35.508264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa636a-67d5-4cda-8bfb-38cf262e2df9 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:35.508548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa636a-67d5-4cda-8bfb-38cf262e2df9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:35.508830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa636a-67d5-4cda-8bfb-38cf262e2df9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:35.519858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa636a-67d5-4cda-8bfb-38cf262e2df9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:05:35.521361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa636a-67d5-4cda-8bfb-38cf262e2df9 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:05:35.532179 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 771/1539] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:05:45.171665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=11 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:05:45.527067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:07.301272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c93ad-e5d9-48ca-86d2-d6c0e637264b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.302616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c93ad-e5d9-48ca-86d2-d6c0e637264b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.303324 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c93ad-e5d9-48ca-86d2-d6c0e637264b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.336494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-056c93ad-e5d9-48ca-86d2-d6c0e637264b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:07.338923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-056c93ad-e5d9-48ca-86d2-d6c0e637264b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:07.351429 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 769/1540] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:07.360913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3ce3c-ee82-4d2c-80e2-75fe50394c84 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.362789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3ce3c-ee82-4d2c-80e2-75fe50394c84 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.362789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3ce3c-ee82-4d2c-80e2-75fe50394c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.384958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-92f3ce3c-ee82-4d2c-80e2-75fe50394c84 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:07.386852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-92f3ce3c-ee82-4d2c-80e2-75fe50394c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:07.399683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 772/1541] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:07.405290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c93ad-e5d9-48ca-86d2-d6c0e637264b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:07.416987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c93ad-e5d9-48ca-86d2-d6c0e637264b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:07.423006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374d3e-e961-4da5-89cd-484c64081b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:07.428401 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374d3e-e961-4da5-89cd-484c64081b33 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.429114 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374d3e-e961-4da5-89cd-484c64081b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.429646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374d3e-e961-4da5-89cd-484c64081b33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.430940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374d3e-e961-4da5-89cd-484c64081b33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:07.431553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374d3e-e961-4da5-89cd-484c64081b33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:07.433576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374d3e-e961-4da5-89cd-484c64081b33 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:07.448063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 770/1542] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:07.456178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3ce3c-ee82-4d2c-80e2-75fe50394c84 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:07.467948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3ce3c-ee82-4d2c-80e2-75fe50394c84 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:07.473595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115282e-366a-4dd8-b3fe-9454420b4300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:07.474742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115282e-366a-4dd8-b3fe-9454420b4300 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.475282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115282e-366a-4dd8-b3fe-9454420b4300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.475827 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115282e-366a-4dd8-b3fe-9454420b4300 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.476834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115282e-366a-4dd8-b3fe-9454420b4300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:07.479147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115282e-366a-4dd8-b3fe-9454420b4300 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:07.491351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c61d9-48a2-4888-b4ed-448581a300a5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.492077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c61d9-48a2-4888-b4ed-448581a300a5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.492652 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c61d9-48a2-4888-b4ed-448581a300a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.533848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c30c61d9-48a2-4888-b4ed-448581a300a5 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:07.540269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c30c61d9-48a2-4888-b4ed-448581a300a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:07.554485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 771/1543] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:07.564420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a9e05-d43d-4793-9590-773459bcab13 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.565072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a9e05-d43d-4793-9590-773459bcab13 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.565574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a9e05-d43d-4793-9590-773459bcab13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.582710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 773/1544] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:07 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:07.587105 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115282e-366a-4dd8-b3fe-9454420b4300 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:07.608670 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8e8a9e05-d43d-4793-9590-773459bcab13 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:07.612863 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8e8a9e05-d43d-4793-9590-773459bcab13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:07.728423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115282e-366a-4dd8-b3fe-9454420b4300 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:07.734125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 772/1545] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:07.736819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e4a8b-f823-428f-8b78-c9ac20fbfa15 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.737380 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e4a8b-f823-428f-8b78-c9ac20fbfa15 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.737875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e4a8b-f823-428f-8b78-c9ac20fbfa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.739557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202a21a-cb6f-4580-81d7-1200b14140d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:07.740251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202a21a-cb6f-4580-81d7-1200b14140d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.740587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202a21a-cb6f-4580-81d7-1200b14140d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.740919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202a21a-cb6f-4580-81d7-1200b14140d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.741617 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202a21a-cb6f-4580-81d7-1200b14140d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:07.742741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-6202a21a-cb6f-4580-81d7-1200b14140d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:07:07.742741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 14 11:07:07.742741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: Jul 14 11:07:07.744028 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202a21a-cb6f-4580-81d7-1200b14140d0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:07.769108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-487e4a8b-f823-428f-8b78-c9ac20fbfa15 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:07.770826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-487e4a8b-f823-428f-8b78-c9ac20fbfa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:07.786352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 773/1546] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:07.787715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e4a8b-f823-428f-8b78-c9ac20fbfa15 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:07.790311 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 774/1547] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:07 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:07.795008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202a21a-cb6f-4580-81d7-1200b14140d0 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:07.801320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e4a8b-f823-428f-8b78-c9ac20fbfa15 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:07.812683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9540632-8b40-4bb4-8924-c7a3e885d214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:07.813791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9540632-8b40-4bb4-8924-c7a3e885d214 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.814303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9540632-8b40-4bb4-8924-c7a3e885d214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.814802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9540632-8b40-4bb4-8924-c7a3e885d214 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.815799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9540632-8b40-4bb4-8924-c7a3e885d214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:07.816341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9540632-8b40-4bb4-8924-c7a3e885d214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:07.818202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9540632-8b40-4bb4-8924-c7a3e885d214 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:07.818334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202a21a-cb6f-4580-81d7-1200b14140d0 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:07.835030 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16f7bf-7f6d-4521-926b-0689286f7660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:07.835866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16f7bf-7f6d-4521-926b-0689286f7660 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.836203 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16f7bf-7f6d-4521-926b-0689286f7660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.836486 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16f7bf-7f6d-4521-926b-0689286f7660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.837261 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16f7bf-7f6d-4521-926b-0689286f7660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:07.838560 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 774/1548] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:07.840200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd2200-19ad-4fa9-8204-a620df2f6e21 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.840774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd2200-19ad-4fa9-8204-a620df2f6e21 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.841294 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd2200-19ad-4fa9-8204-a620df2f6e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.844871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16f7bf-7f6d-4521-926b-0689286f7660 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:07.861727 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0cdd2200-19ad-4fa9-8204-a620df2f6e21 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:07.863335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0cdd2200-19ad-4fa9-8204-a620df2f6e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:07.864666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 775/1549] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:07:07.865565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16f7bf-7f6d-4521-926b-0689286f7660 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:07.881956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 775/1550] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:07.883233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be63c54-dd62-45c0-a6e3-717f1dd6460a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.883596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be63c54-dd62-45c0-a6e3-717f1dd6460a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.883881 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be63c54-dd62-45c0-a6e3-717f1dd6460a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.889179 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16f7bf-7f6d-4521-926b-0689286f7660 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:07.900406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66456c-73bc-46f4-a99e-1f59dc28be20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:07.901041 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66456c-73bc-46f4-a99e-1f59dc28be20 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:07.901337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66456c-73bc-46f4-a99e-1f59dc28be20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:07.901618 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66456c-73bc-46f4-a99e-1f59dc28be20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:07.902345 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66456c-73bc-46f4-a99e-1f59dc28be20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:07.904870 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66456c-73bc-46f4-a99e-1f59dc28be20 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:07.929445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-3be63c54-dd62-45c0-a6e3-717f1dd6460a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:07.931173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3be63c54-dd62-45c0-a6e3-717f1dd6460a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:07.953661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 776/1551] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:07.954560 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be63c54-dd62-45c0-a6e3-717f1dd6460a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:07.968849 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 776/1552] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:07 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:07.969802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66456c-73bc-46f4-a99e-1f59dc28be20 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:07.970046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be63c54-dd62-45c0-a6e3-717f1dd6460a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.042278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3edc77-b442-4a00-8ff6-ceb85453daf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.044499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3edc77-b442-4a00-8ff6-ceb85453daf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.044791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3edc77-b442-4a00-8ff6-ceb85453daf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.045075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3edc77-b442-4a00-8ff6-ceb85453daf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.046205 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3edc77-b442-4a00-8ff6-ceb85453daf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.047838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66456c-73bc-46f4-a99e-1f59dc28be20 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.048097 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3edc77-b442-4a00-8ff6-ceb85453daf4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.053014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade35029-07aa-43ac-94a5-7224bc15398b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.060443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade35029-07aa-43ac-94a5-7224bc15398b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.060443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade35029-07aa-43ac-94a5-7224bc15398b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.060443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade35029-07aa-43ac-94a5-7224bc15398b admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.060443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade35029-07aa-43ac-94a5-7224bc15398b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:08.060443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade35029-07aa-43ac-94a5-7224bc15398b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.061351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-ade35029-07aa-43ac-94a5-7224bc15398b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:07:08.061351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 14 11:07:08.061351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: Jul 14 11:07:08.061351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade35029-07aa-43ac-94a5-7224bc15398b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.061927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 777/1553] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:08.061927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6a015-d470-4efb-ab6a-4847a3d10bfd None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.061927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6a015-d470-4efb-ab6a-4847a3d10bfd None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.062388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6a015-d470-4efb-ab6a-4847a3d10bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.073034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 777/1554] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:08.074374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfb8c8-85a4-432d-90bd-9fa833d81682 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.074749 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfb8c8-85a4-432d-90bd-9fa833d81682 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.075042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfb8c8-85a4-432d-90bd-9fa833d81682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.089539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-87e6a015-d470-4efb-ab6a-4847a3d10bfd None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:08.091305 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-87e6a015-d470-4efb-ab6a-4847a3d10bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:08.095879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-47dfb8c8-85a4-432d-90bd-9fa833d81682 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:08.097310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-47dfb8c8-85a4-432d-90bd-9fa833d81682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:08.101082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 778/1555] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:08.102804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65058470-5a48-44a4-874a-c3ac0a99a085 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.103192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65058470-5a48-44a4-874a-c3ac0a99a085 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.103517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65058470-5a48-44a4-874a-c3ac0a99a085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.108138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 778/1556] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:08.109184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfb8c8-85a4-432d-90bd-9fa833d81682 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.119587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfb8c8-85a4-432d-90bd-9fa833d81682 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.124656 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d18ea6-d64b-4855-be2b-60cc050890da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.125416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d18ea6-d64b-4855-be2b-60cc050890da admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.125707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d18ea6-d64b-4855-be2b-60cc050890da admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.126092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d18ea6-d64b-4855-be2b-60cc050890da admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.126794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d18ea6-d64b-4855-be2b-60cc050890da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.128781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d18ea6-d64b-4855-be2b-60cc050890da admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.129646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-65058470-5a48-44a4-874a-c3ac0a99a085 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:08.131507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-65058470-5a48-44a4-874a-c3ac0a99a085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:08.136367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:08.144018 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 779/1557] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:08.148252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39268a-c2fe-4d5c-88a7-7e190310b044 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.148252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39268a-c2fe-4d5c-88a7-7e190310b044 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.148252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39268a-c2fe-4d5c-88a7-7e190310b044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.158586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 779/1558] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/users => generated 292 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:08.158586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404ff62-48e1-4b9d-8352-a48385da8121 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.158586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404ff62-48e1-4b9d-8352-a48385da8121 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.158586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404ff62-48e1-4b9d-8352-a48385da8121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.170571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fa39268a-c2fe-4d5c-88a7-7e190310b044 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:08.171970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fa39268a-c2fe-4d5c-88a7-7e190310b044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:08.177903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8404ff62-48e1-4b9d-8352-a48385da8121 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:08.179104 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8404ff62-48e1-4b9d-8352-a48385da8121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:08.192001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 780/1559] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:08.192001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39268a-c2fe-4d5c-88a7-7e190310b044 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.197225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39268a-c2fe-4d5c-88a7-7e190310b044 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.202438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9bdf0-0737-4b0a-9e15-19b9b2d26c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.203569 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 780/1560] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:08.203800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9bdf0-0737-4b0a-9e15-19b9b2d26c56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.203921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9bdf0-0737-4b0a-9e15-19b9b2d26c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.204176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9bdf0-0737-4b0a-9e15-19b9b2d26c56 admin admin] PATH_INFO: `/v3/projects/e4a8af112fa744a185f2dfe34224f554/users/e4b3fb9e24a24db7afb30583737f1314/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.205362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404ff62-48e1-4b9d-8352-a48385da8121 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.217895 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404ff62-48e1-4b9d-8352-a48385da8121 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.222998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9bdf0-0737-4b0a-9e15-19b9b2d26c56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a8af112fa744a185f2dfe34224f554, user_id=e4b3fb9e24a24db7afb30583737f1314, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.224038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc63b8e-c5df-4ea2-9a6f-8f13b7c72a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.224567 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-4cb9bdf0-0737-4b0a-9e15-19b9b2d26c56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:07:08.224567 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 14 11:07:08.224567 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: Jul 14 11:07:08.225057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc63b8e-c5df-4ea2-9a6f-8f13b7c72a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.225239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc63b8e-c5df-4ea2-9a6f-8f13b7c72a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.225450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc63b8e-c5df-4ea2-9a6f-8f13b7c72a41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.226294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc63b8e-c5df-4ea2-9a6f-8f13b7c72a41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.227278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9bdf0-0737-4b0a-9e15-19b9b2d26c56 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.228338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc63b8e-c5df-4ea2-9a6f-8f13b7c72a41 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.230377 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:08.244745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 781/1561] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:08 2022] PUT /identity/v3/projects/e4a8af112fa744a185f2dfe34224f554/users/e4b3fb9e24a24db7afb30583737f1314/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:08.246212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601e28e-8d44-4614-af91-32e225ac5524 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.246378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601e28e-8d44-4614-af91-32e225ac5524 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.246637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601e28e-8d44-4614-af91-32e225ac5524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.264560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 781/1562] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:08.265915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ee4fa-5963-4504-afc5-2af796187058 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.266215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ee4fa-5963-4504-afc5-2af796187058 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.266502 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ee4fa-5963-4504-afc5-2af796187058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.274060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9601e28e-8d44-4614-af91-32e225ac5524 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:08.275512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9601e28e-8d44-4614-af91-32e225ac5524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:08.286261 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b11ee4fa-5963-4504-afc5-2af796187058 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:08.287502 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b11ee4fa-5963-4504-afc5-2af796187058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:08.355134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 782/1563] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:08.356124 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601e28e-8d44-4614-af91-32e225ac5524 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.366241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601e28e-8d44-4614-af91-32e225ac5524 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.367852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 782/1564] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:08.369611 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ee4fa-5963-4504-afc5-2af796187058 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.375795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6991d83-a74a-47f9-aacd-fdf35d97d1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.376699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6991d83-a74a-47f9-aacd-fdf35d97d1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.377084 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6991d83-a74a-47f9-aacd-fdf35d97d1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.377411 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6991d83-a74a-47f9-aacd-fdf35d97d1f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.378281 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6991d83-a74a-47f9-aacd-fdf35d97d1f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:08.378590 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6991d83-a74a-47f9-aacd-fdf35d97d1f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.380596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6991d83-a74a-47f9-aacd-fdf35d97d1f4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.382197 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ee4fa-5963-4504-afc5-2af796187058 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.388745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347a4a4-9bdb-4f98-8552-3217c6b247e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.389558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347a4a4-9bdb-4f98-8552-3217c6b247e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.389872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347a4a4-9bdb-4f98-8552-3217c6b247e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.390235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347a4a4-9bdb-4f98-8552-3217c6b247e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.391113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347a4a4-9bdb-4f98-8552-3217c6b247e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.393109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347a4a4-9bdb-4f98-8552-3217c6b247e4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.394060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 783/1565] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:08.395399 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fa071-0540-4f0a-b209-5173bf22ebc2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.395705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fa071-0540-4f0a-b209-5173bf22ebc2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.395983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fa071-0540-4f0a-b209-5173bf22ebc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.405575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 783/1566] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:08.409063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8f81a-3b7a-46e8-85d7-ad82e310c8e7 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.409365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8f81a-3b7a-46e8-85d7-ad82e310c8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.409653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8f81a-3b7a-46e8-85d7-ad82e310c8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.415556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b91fa071-0540-4f0a-b209-5173bf22ebc2 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:08.416775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b91fa071-0540-4f0a-b209-5173bf22ebc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:08.426401 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 784/1567] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:08.427739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9e332-e161-4613-a9ab-c6d68c72ecb3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.428031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9e332-e161-4613-a9ab-c6d68c72ecb3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.428310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9e332-e161-4613-a9ab-c6d68c72ecb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.432664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-24b8f81a-3b7a-46e8-85d7-ad82e310c8e7 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:08.434283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-24b8f81a-3b7a-46e8-85d7-ad82e310c8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:08.443047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 784/1568] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:08.444737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8f81a-3b7a-46e8-85d7-ad82e310c8e7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.455929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8f81a-3b7a-46e8-85d7-ad82e310c8e7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.461170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4f3a6-7977-4607-98fe-267ac6770015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.461686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-cbf9e332-e161-4613-a9ab-c6d68c72ecb3 None None] MFA Rules not processed for user `e4b3fb9e24a24db7afb30583737f1314`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:08.461861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4f3a6-7977-4607-98fe-267ac6770015 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.462153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4f3a6-7977-4607-98fe-267ac6770015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.462437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4f3a6-7977-4607-98fe-267ac6770015 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.462926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cbf9e332-e161-4613-a9ab-c6d68c72ecb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:08.463141 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4f3a6-7977-4607-98fe-267ac6770015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.466045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4f3a6-7977-4607-98fe-267ac6770015 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.487798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 785/1569] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:08.488729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4f3a6-7977-4607-98fe-267ac6770015 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.501469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4f3a6-7977-4607-98fe-267ac6770015 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.507119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498c204-c9d6-4cae-8735-115856277253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.507983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498c204-c9d6-4cae-8735-115856277253 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.508274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498c204-c9d6-4cae-8735-115856277253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.508585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498c204-c9d6-4cae-8735-115856277253 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.509330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498c204-c9d6-4cae-8735-115856277253 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:08.509656 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498c204-c9d6-4cae-8735-115856277253 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.511277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498c204-c9d6-4cae-8735-115856277253 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.527489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 786/1570] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:08.527489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498c204-c9d6-4cae-8735-115856277253 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.541180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498c204-c9d6-4cae-8735-115856277253 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.546023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0ec6-5bbf-4595-a006-5ce7567f9cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.546815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0ec6-5bbf-4595-a006-5ce7567f9cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.547362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0ec6-5bbf-4595-a006-5ce7567f9cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.547639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0ec6-5bbf-4595-a006-5ce7567f9cf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.548442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe0ec6-5bbf-4595-a006-5ce7567f9cf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:08.548701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe0ec6-5bbf-4595-a006-5ce7567f9cf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.550351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe0ec6-5bbf-4595-a006-5ce7567f9cf7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.562273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 785/1571] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:08.567568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9e332-e161-4613-a9ab-c6d68c72ecb3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.573101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 787/1572] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:08.574465 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0ec6-5bbf-4595-a006-5ce7567f9cf7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.586220 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9e332-e161-4613-a9ab-c6d68c72ecb3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.586645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0ec6-5bbf-4595-a006-5ce7567f9cf7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.594652 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad2e66-e609-4823-aaf3-6022e2bd3b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.594907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad2e66-e609-4823-aaf3-6022e2bd3b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.595098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad2e66-e609-4823-aaf3-6022e2bd3b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.595969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad2e66-e609-4823-aaf3-6022e2bd3b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.596541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad2e66-e609-4823-aaf3-6022e2bd3b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.598542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-93ad2e66-e609-4823-aaf3-6022e2bd3b45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:07:08.598542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 14 11:07:08.598542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: Jul 14 11:07:08.600702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad2e66-e609-4823-aaf3-6022e2bd3b45 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.602032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0583b46-9e28-477c-ba55-56188a951020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.603216 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0583b46-9e28-477c-ba55-56188a951020 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.603855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0583b46-9e28-477c-ba55-56188a951020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.604330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0583b46-9e28-477c-ba55-56188a951020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.605497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0583b46-9e28-477c-ba55-56188a951020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.608969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0583b46-9e28-477c-ba55-56188a951020 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.629806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 788/1573] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:08.631223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0583b46-9e28-477c-ba55-56188a951020 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.633805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 786/1574] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:08.634662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad2e66-e609-4823-aaf3-6022e2bd3b45 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.649408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0583b46-9e28-477c-ba55-56188a951020 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.695065 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:08.710714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:08.717246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad2e66-e609-4823-aaf3-6022e2bd3b45 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.722553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1625e9-8809-4448-b4bb-e725ff82e7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.723233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1625e9-8809-4448-b4bb-e725ff82e7aa admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.723550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1625e9-8809-4448-b4bb-e725ff82e7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.723840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1625e9-8809-4448-b4bb-e725ff82e7aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.724560 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1625e9-8809-4448-b4bb-e725ff82e7aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:08.724873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1625e9-8809-4448-b4bb-e725ff82e7aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.726501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1625e9-8809-4448-b4bb-e725ff82e7aa admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.728198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09b0d9-f038-491e-a6d1-f5fa6e6ad7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.729270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09b0d9-f038-491e-a6d1-f5fa6e6ad7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.729828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09b0d9-f038-491e-a6d1-f5fa6e6ad7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.730302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09b0d9-f038-491e-a6d1-f5fa6e6ad7d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.731362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09b0d9-f038-491e-a6d1-f5fa6e6ad7d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:08.731752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09b0d9-f038-491e-a6d1-f5fa6e6ad7d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.734259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09b0d9-f038-491e-a6d1-f5fa6e6ad7d8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.743523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 787/1575] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:08.743523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1625e9-8809-4448-b4bb-e725ff82e7aa admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.755468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1625e9-8809-4448-b4bb-e725ff82e7aa admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.758189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 789/1576] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:08.761085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09b0d9-f038-491e-a6d1-f5fa6e6ad7d8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.765109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74699809-4ca0-43f7-973d-816a6be961ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.766478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74699809-4ca0-43f7-973d-816a6be961ef admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.767023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74699809-4ca0-43f7-973d-816a6be961ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.767547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74699809-4ca0-43f7-973d-816a6be961ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.768876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74699809-4ca0-43f7-973d-816a6be961ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.771210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74699809-4ca0-43f7-973d-816a6be961ef admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.779572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09b0d9-f038-491e-a6d1-f5fa6e6ad7d8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.784739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35747505-5c7a-48ea-9546-9ed8a5897775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.785411 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35747505-5c7a-48ea-9546-9ed8a5897775 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.785694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35747505-5c7a-48ea-9546-9ed8a5897775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.789149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 788/1577] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:08.789294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35747505-5c7a-48ea-9546-9ed8a5897775 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.789378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74699809-4ca0-43f7-973d-816a6be961ef admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.789476 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35747505-5c7a-48ea-9546-9ed8a5897775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.789476 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35747505-5c7a-48ea-9546-9ed8a5897775 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.799364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74699809-4ca0-43f7-973d-816a6be961ef admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.805933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61e463-1707-4f97-8b27-40482f06abd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.806530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61e463-1707-4f97-8b27-40482f06abd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.806822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61e463-1707-4f97-8b27-40482f06abd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.807102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61e463-1707-4f97-8b27-40482f06abd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.807891 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61e463-1707-4f97-8b27-40482f06abd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.810373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61e463-1707-4f97-8b27-40482f06abd6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.827415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 790/1578] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:08.829230 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0d13e-4640-4a64-ac3c-4cd7cbbbe29b None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.829523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0d13e-4640-4a64-ac3c-4cd7cbbbe29b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.829872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0d13e-4640-4a64-ac3c-4cd7cbbbe29b None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.831296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 791/1579] 10.222.0.75 () {54 vars in 961 bytes} [Thu Jul 14 11:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:07:08.832089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0d13e-4640-4a64-ac3c-4cd7cbbbe29b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.848574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 789/1580] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:08.849554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61e463-1707-4f97-8b27-40482f06abd6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.911808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0d13e-4640-4a64-ac3c-4cd7cbbbe29b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.933909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61e463-1707-4f97-8b27-40482f06abd6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.939578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43511f1-5485-4e2a-bf87-d0cc318b195b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.940272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43511f1-5485-4e2a-bf87-d0cc318b195b admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.940558 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43511f1-5485-4e2a-bf87-d0cc318b195b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.940837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43511f1-5485-4e2a-bf87-d0cc318b195b admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.941536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43511f1-5485-4e2a-bf87-d0cc318b195b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.946047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43511f1-5485-4e2a-bf87-d0cc318b195b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:08.969024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 790/1581] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:08.969847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43511f1-5485-4e2a-bf87-d0cc318b195b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:08.980365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43511f1-5485-4e2a-bf87-d0cc318b195b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:08.985349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47556706-305e-4fb4-9fe8-3422fb51f46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.985961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47556706-305e-4fb4-9fe8-3422fb51f46d admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.986111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea3917-bd92-4259-9f7d-08268d9bed06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:08.986247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47556706-305e-4fb4-9fe8-3422fb51f46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.986517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47556706-305e-4fb4-9fe8-3422fb51f46d admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.987275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47556706-305e-4fb4-9fe8-3422fb51f46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:08.987408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea3917-bd92-4259-9f7d-08268d9bed06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:08.987937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea3917-bd92-4259-9f7d-08268d9bed06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:08.988229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea3917-bd92-4259-9f7d-08268d9bed06 admin admin] PATH_INFO: `/v3/projects/88118eaad2474baa9a9039d13a705efe/users/0b4caa892c3c4a3fb9ee06e6870f7160/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:08.989251 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47556706-305e-4fb4-9fe8-3422fb51f46d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.009631 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea3917-bd92-4259-9f7d-08268d9bed06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88118eaad2474baa9a9039d13a705efe, user_id=0b4caa892c3c4a3fb9ee06e6870f7160, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.021053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea3917-bd92-4259-9f7d-08268d9bed06 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.025728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 791/1582] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:08 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:09.025728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47556706-305e-4fb4-9fe8-3422fb51f46d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.040775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 792/1583] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:08 2022] PUT /identity/v3/projects/88118eaad2474baa9a9039d13a705efe/users/0b4caa892c3c4a3fb9ee06e6870f7160/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:09.041717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea3917-bd92-4259-9f7d-08268d9bed06 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.093413 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47556706-305e-4fb4-9fe8-3422fb51f46d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.098945 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:09.109598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:09.112685 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea3917-bd92-4259-9f7d-08268d9bed06 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.118081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7e46d-8dfe-497e-a509-60706c09b4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.118771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7e46d-8dfe-497e-a509-60706c09b4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.119056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7e46d-8dfe-497e-a509-60706c09b4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.119333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7e46d-8dfe-497e-a509-60706c09b4a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.120093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7e46d-8dfe-497e-a509-60706c09b4a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.122313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7e46d-8dfe-497e-a509-60706c09b4a3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.133186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 793/1584] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:09.133865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7e46d-8dfe-497e-a509-60706c09b4a3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.144674 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7e46d-8dfe-497e-a509-60706c09b4a3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.150163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd78bb9-0e56-425d-9d77-e6ff641c14c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.150829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd78bb9-0e56-425d-9d77-e6ff641c14c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.151192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd78bb9-0e56-425d-9d77-e6ff641c14c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.151519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd78bb9-0e56-425d-9d77-e6ff641c14c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.152211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd78bb9-0e56-425d-9d77-e6ff641c14c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.154250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd78bb9-0e56-425d-9d77-e6ff641c14c6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.169343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc192410-b4f7-461f-88df-dad04c4bb71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.169935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc192410-b4f7-461f-88df-dad04c4bb71a admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.170229 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc192410-b4f7-461f-88df-dad04c4bb71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.170583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc192410-b4f7-461f-88df-dad04c4bb71a admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.171342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc192410-b4f7-461f-88df-dad04c4bb71a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.173335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc192410-b4f7-461f-88df-dad04c4bb71a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.181816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 794/1585] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:09 2022] POST /identity/v3/users => generated 288 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:09.181816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691041b-1ea7-42d3-a4e0-42c62f730807 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.181816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691041b-1ea7-42d3-a4e0-42c62f730807 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.181816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691041b-1ea7-42d3-a4e0-42c62f730807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.218661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 792/1586] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:09 2022] POST /identity/v3/projects => generated 391 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:09.220650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc192410-b4f7-461f-88df-dad04c4bb71a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.240801 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f691041b-1ea7-42d3-a4e0-42c62f730807 None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:09.242034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f691041b-1ea7-42d3-a4e0-42c62f730807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:09.296717 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc192410-b4f7-461f-88df-dad04c4bb71a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.301792 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead609e1-39a5-4a79-85fc-eaf5f0bee2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.302476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead609e1-39a5-4a79-85fc-eaf5f0bee2be admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.302764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead609e1-39a5-4a79-85fc-eaf5f0bee2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.303044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead609e1-39a5-4a79-85fc-eaf5f0bee2be admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.303782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead609e1-39a5-4a79-85fc-eaf5f0bee2be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.306002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead609e1-39a5-4a79-85fc-eaf5f0bee2be admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.332033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 793/1587] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:09 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:09.333150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead609e1-39a5-4a79-85fc-eaf5f0bee2be admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.343677 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead609e1-39a5-4a79-85fc-eaf5f0bee2be admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.348363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c7a5b-d7fa-4959-9b94-7d30929cda23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.349029 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c7a5b-d7fa-4959-9b94-7d30929cda23 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.349320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c7a5b-d7fa-4959-9b94-7d30929cda23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.349602 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c7a5b-d7fa-4959-9b94-7d30929cda23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.350363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c7a5b-d7fa-4959-9b94-7d30929cda23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.352040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c7a5b-d7fa-4959-9b94-7d30929cda23 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.352903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 795/1588] 10.222.0.75 () {58 vars in 1058 bytes} [Thu Jul 14 11:07:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:09.355372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691041b-1ea7-42d3-a4e0-42c62f730807 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.366409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 794/1589] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:09.366409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff4037-18ac-4f24-ad59-345a08b833e2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.366409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff4037-18ac-4f24-ad59-345a08b833e2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.366409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff4037-18ac-4f24-ad59-345a08b833e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.375577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691041b-1ea7-42d3-a4e0-42c62f730807 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.383026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dceaa4-a4f2-4050-870f-71427324a7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.384270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dceaa4-a4f2-4050-870f-71427324a7ec admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.384741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dceaa4-a4f2-4050-870f-71427324a7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.385250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dceaa4-a4f2-4050-870f-71427324a7ec admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.386752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dceaa4-a4f2-4050-870f-71427324a7ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.390757 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dceaa4-a4f2-4050-870f-71427324a7ec admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.396146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7fff4037-18ac-4f24-ad59-345a08b833e2 None None] MFA Rules not processed for user `0b4caa892c3c4a3fb9ee06e6870f7160`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:09.397420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7fff4037-18ac-4f24-ad59-345a08b833e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:09.423822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 796/1590] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:09 2022] POST /identity/v3/users => generated 295 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:09.425241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dceaa4-a4f2-4050-870f-71427324a7ec admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.440234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dceaa4-a4f2-4050-870f-71427324a7ec admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.446661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415349f-da50-4577-85f2-4f5ccca2bac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.447992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415349f-da50-4577-85f2-4f5ccca2bac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.448430 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415349f-da50-4577-85f2-4f5ccca2bac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.448854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415349f-da50-4577-85f2-4f5ccca2bac5 admin admin] PATH_INFO: `/v3/projects/7f04452d94114c64b9882cfa66e29479/users/fffbf852ddc9455f8a68f80ad118e4d1/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.468361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415349f-da50-4577-85f2-4f5ccca2bac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f04452d94114c64b9882cfa66e29479, user_id=fffbf852ddc9455f8a68f80ad118e4d1, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.471232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415349f-da50-4577-85f2-4f5ccca2bac5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.472547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 795/1591] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:09.473271 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff4037-18ac-4f24-ad59-345a08b833e2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.489386 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 797/1592] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:09 2022] PUT /identity/v3/projects/7f04452d94114c64b9882cfa66e29479/users/fffbf852ddc9455f8a68f80ad118e4d1/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:09.490321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff4037-18ac-4f24-ad59-345a08b833e2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.490450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415349f-da50-4577-85f2-4f5ccca2bac5 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.507005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:09.574901 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c512b0-206b-406f-8526-2870e59cb9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.576012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c512b0-206b-406f-8526-2870e59cb9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.576545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c512b0-206b-406f-8526-2870e59cb9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.577037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c512b0-206b-406f-8526-2870e59cb9e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.578182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c512b0-206b-406f-8526-2870e59cb9e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.580263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c512b0-206b-406f-8526-2870e59cb9e8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.580830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415349f-da50-4577-85f2-4f5ccca2bac5 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.587916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337750e1-3dfd-49e6-b207-3322c0fed72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.588623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337750e1-3dfd-49e6-b207-3322c0fed72a admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.588905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337750e1-3dfd-49e6-b207-3322c0fed72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.589195 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337750e1-3dfd-49e6-b207-3322c0fed72a admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.589904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337750e1-3dfd-49e6-b207-3322c0fed72a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.592486 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337750e1-3dfd-49e6-b207-3322c0fed72a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.601091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 796/1593] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:09.602512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c512b0-206b-406f-8526-2870e59cb9e8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.604674 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:09.617051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c512b0-206b-406f-8526-2870e59cb9e8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.621934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739943a-f3a2-4dcc-b640-be89f6a9f629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.622201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 798/1594] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:09 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:09.622759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739943a-f3a2-4dcc-b640-be89f6a9f629 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.623061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739943a-f3a2-4dcc-b640-be89f6a9f629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.623470 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739943a-f3a2-4dcc-b640-be89f6a9f629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.623691 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337750e1-3dfd-49e6-b207-3322c0fed72a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.624229 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739943a-f3a2-4dcc-b640-be89f6a9f629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.625943 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739943a-f3a2-4dcc-b640-be89f6a9f629 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.644913 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 797/1595] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:09.646118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739943a-f3a2-4dcc-b640-be89f6a9f629 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.662113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739943a-f3a2-4dcc-b640-be89f6a9f629 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.670155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02da7f-72b3-48ec-8a92-86aa4eb7e1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.671282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02da7f-72b3-48ec-8a92-86aa4eb7e1ac admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.671798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02da7f-72b3-48ec-8a92-86aa4eb7e1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.672297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02da7f-72b3-48ec-8a92-86aa4eb7e1ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.673457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02da7f-72b3-48ec-8a92-86aa4eb7e1ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.676160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02da7f-72b3-48ec-8a92-86aa4eb7e1ac admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.692631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 798/1596] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:09.693645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02da7f-72b3-48ec-8a92-86aa4eb7e1ac admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.707139 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02da7f-72b3-48ec-8a92-86aa4eb7e1ac admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.714900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2970933-2fa2-412b-a095-32793fe76ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.716845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2970933-2fa2-412b-a095-32793fe76ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.717060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2970933-2fa2-412b-a095-32793fe76ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.717540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2970933-2fa2-412b-a095-32793fe76ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.718897 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2970933-2fa2-412b-a095-32793fe76ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.721990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2970933-2fa2-412b-a095-32793fe76ce8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.722955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337750e1-3dfd-49e6-b207-3322c0fed72a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.729432 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0416d3a-616b-4a2a-baf8-6209e914d822 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.730102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0416d3a-616b-4a2a-baf8-6209e914d822 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.730387 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0416d3a-616b-4a2a-baf8-6209e914d822 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.730714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0416d3a-616b-4a2a-baf8-6209e914d822 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.741736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 799/1597] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:09.744145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8090a8c-3332-4189-948b-f03ceab592e2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.744746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8090a8c-3332-4189-948b-f03ceab592e2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.745226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8090a8c-3332-4189-948b-f03ceab592e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.772121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a8090a8c-3332-4189-948b-f03ceab592e2 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:09.773791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a8090a8c-3332-4189-948b-f03ceab592e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:09.785664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 800/1598] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:09.789336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e060dc-b3b6-481b-a34c-f58028522395 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.789336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e060dc-b3b6-481b-a34c-f58028522395 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.789336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e060dc-b3b6-481b-a34c-f58028522395 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.790415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 801/1599] 10.222.0.75 () {54 vars in 961 bytes} [Thu Jul 14 11:07:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:07:09.791554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e060dc-b3b6-481b-a34c-f58028522395 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.802545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0416d3a-616b-4a2a-baf8-6209e914d822 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.804985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0416d3a-616b-4a2a-baf8-6209e914d822 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.811647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e060dc-b3b6-481b-a34c-f58028522395 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.816110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 799/1600] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:09.816726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0416d3a-616b-4a2a-baf8-6209e914d822 service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.819940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4c6c2-84e6-4e8c-a3ab-88a4a83173d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.821047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4c6c2-84e6-4e8c-a3ab-88a4a83173d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.821552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4c6c2-84e6-4e8c-a3ab-88a4a83173d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.821997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4c6c2-84e6-4e8c-a3ab-88a4a83173d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.823298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4c6c2-84e6-4e8c-a3ab-88a4a83173d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.826644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4c6c2-84e6-4e8c-a3ab-88a4a83173d8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.830142 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0416d3a-616b-4a2a-baf8-6209e914d822 service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.836415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6d35a-6a79-4606-a34d-81bea701c064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.837169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6d35a-6a79-4606-a34d-81bea701c064 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.837467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6d35a-6a79-4606-a34d-81bea701c064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.837736 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6d35a-6a79-4606-a34d-81bea701c064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.838532 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6d35a-6a79-4606-a34d-81bea701c064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.841052 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6d35a-6a79-4606-a34d-81bea701c064 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.846315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 802/1601] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:09.849964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4c6c2-84e6-4e8c-a3ab-88a4a83173d8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.854120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 800/1602] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:09.855232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6d35a-6a79-4606-a34d-81bea701c064 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.866991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6d35a-6a79-4606-a34d-81bea701c064 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.871783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4c6c2-84e6-4e8c-a3ab-88a4a83173d8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.872210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68499c2e-3eee-4b47-ab04-f86a33dc9215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.873137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68499c2e-3eee-4b47-ab04-f86a33dc9215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.873595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68499c2e-3eee-4b47-ab04-f86a33dc9215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.873998 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68499c2e-3eee-4b47-ab04-f86a33dc9215 admin admin] PATH_INFO: `/v3/projects/02935d215ec341e189241e3a6c9e2747/users/ea9bcb761c7a4655a35da1948d35ef4c/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.882062 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c77ebb-2cb5-4578-b959-cc18a3678e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:09.883336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c77ebb-2cb5-4578-b959-cc18a3678e7e admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.883875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c77ebb-2cb5-4578-b959-cc18a3678e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.885130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c77ebb-2cb5-4578-b959-cc18a3678e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.886433 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c77ebb-2cb5-4578-b959-cc18a3678e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.890129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c77ebb-2cb5-4578-b959-cc18a3678e7e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.893727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68499c2e-3eee-4b47-ab04-f86a33dc9215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02935d215ec341e189241e3a6c9e2747, user_id=ea9bcb761c7a4655a35da1948d35ef4c, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:09.896502 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68499c2e-3eee-4b47-ab04-f86a33dc9215 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:09.911339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 803/1603] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:09.911339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c77ebb-2cb5-4578-b959-cc18a3678e7e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:09.920723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 801/1604] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:09 2022] PUT /identity/v3/projects/02935d215ec341e189241e3a6c9e2747/users/ea9bcb761c7a4655a35da1948d35ef4c/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:09.922508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a9aa6-8285-4607-87d0-aad3c7c134e7 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:09.923042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a9aa6-8285-4607-87d0-aad3c7c134e7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:09.923604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a9aa6-8285-4607-87d0-aad3c7c134e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:09.936058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c77ebb-2cb5-4578-b959-cc18a3678e7e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:09.948422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a49a9aa6-8285-4607-87d0-aad3c7c134e7 None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:09.950217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a49a9aa6-8285-4607-87d0-aad3c7c134e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:10.024120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:10.046688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e78f5e-39e4-4406-b56a-d6bd01592ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:10.047541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e78f5e-39e4-4406-b56a-d6bd01592ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.047944 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e78f5e-39e4-4406-b56a-d6bd01592ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.048438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e78f5e-39e4-4406-b56a-d6bd01592ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.049578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e78f5e-39e4-4406-b56a-d6bd01592ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:10.050237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 802/1605] 10.222.0.75 () {58 vars in 1058 bytes} [Thu Jul 14 11:07:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:10.051251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a9aa6-8285-4607-87d0-aad3c7c134e7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:10.052782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e78f5e-39e4-4406-b56a-d6bd01592ba8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:10.052987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a49a9aa6-8285-4607-87d0-aad3c7c134e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:10.071611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 804/1606] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:10.071611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e78f5e-39e4-4406-b56a-d6bd01592ba8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:10.084694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e78f5e-39e4-4406-b56a-d6bd01592ba8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:10.091576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e9948-d40c-4d18-9554-ab6ef298adde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:10.092743 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e9948-d40c-4d18-9554-ab6ef298adde admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.093070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e9948-d40c-4d18-9554-ab6ef298adde admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.093366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e9948-d40c-4d18-9554-ab6ef298adde admin admin] PATH_INFO: `/v3/projects/a90a870ee4154287be1282b3f46bdd05/users/f330cdfd7d2b47a985d8dc76c19da9a4/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.111224 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e9948-d40c-4d18-9554-ab6ef298adde admin admin] RBAC: Authorizing `identity:create_grant(project_id=a90a870ee4154287be1282b3f46bdd05, user_id=f330cdfd7d2b47a985d8dc76c19da9a4, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:10.114525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e9948-d40c-4d18-9554-ab6ef298adde admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:10.127548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a9aa6-8285-4607-87d0-aad3c7c134e7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:10.132127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 805/1607] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:10 2022] PUT /identity/v3/projects/a90a870ee4154287be1282b3f46bdd05/users/f330cdfd7d2b47a985d8dc76c19da9a4/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:10.138440 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e9948-d40c-4d18-9554-ab6ef298adde admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:10.156944 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:10.198054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e2ea2-747f-4d62-aeae-19ff8a728aff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:10.199192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e2ea2-747f-4d62-aeae-19ff8a728aff service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.199579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e2ea2-747f-4d62-aeae-19ff8a728aff service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.200085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e2ea2-747f-4d62-aeae-19ff8a728aff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.222969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e9948-d40c-4d18-9554-ab6ef298adde admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:10.229472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbc10a-cb32-4b3a-b04c-d7296e36bfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:10.230937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbc10a-cb32-4b3a-b04c-d7296e36bfd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.232063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbc10a-cb32-4b3a-b04c-d7296e36bfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.232656 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbc10a-cb32-4b3a-b04c-d7296e36bfd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.234035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fbc10a-cb32-4b3a-b04c-d7296e36bfd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:10.237145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fbc10a-cb32-4b3a-b04c-d7296e36bfd4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:10.253593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 806/1608] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:10.254969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbc10a-cb32-4b3a-b04c-d7296e36bfd4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:10.269514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbc10a-cb32-4b3a-b04c-d7296e36bfd4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:10.274880 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fb992-1cfb-47a7-8d26-354a7cf28cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:10.276051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fb992-1cfb-47a7-8d26-354a7cf28cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.276537 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fb992-1cfb-47a7-8d26-354a7cf28cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.277038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fb992-1cfb-47a7-8d26-354a7cf28cc1 admin admin] PATH_INFO: `/v3/projects/b3c9c45844504b80b76887e8450188f8/users/02052e514aaa43d9af166f2bf092d93c/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.281970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e2ea2-747f-4d62-aeae-19ff8a728aff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:10.283643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e2ea2-747f-4d62-aeae-19ff8a728aff service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:10.296243 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 803/1609] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:10.297610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246dc8-fc56-4a4f-b632-d9d84abfc48d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.298248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246dc8-fc56-4a4f-b632-d9d84abfc48d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.298248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246dc8-fc56-4a4f-b632-d9d84abfc48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.302480 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fb992-1cfb-47a7-8d26-354a7cf28cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c9c45844504b80b76887e8450188f8, user_id=02052e514aaa43d9af166f2bf092d93c, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:10.308327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fb992-1cfb-47a7-8d26-354a7cf28cc1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:10.328168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 807/1610] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:10 2022] PUT /identity/v3/projects/b3c9c45844504b80b76887e8450188f8/users/02052e514aaa43d9af166f2bf092d93c/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:10.329128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fb992-1cfb-47a7-8d26-354a7cf28cc1 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:10.345394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-17246dc8-fc56-4a4f-b632-d9d84abfc48d None None] MFA Rules not processed for user `ea9bcb761c7a4655a35da1948d35ef4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:10.347373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-17246dc8-fc56-4a4f-b632-d9d84abfc48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:10.403413 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:10.429001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 804/1611] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:10.433289 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17246dc8-fc56-4a4f-b632-d9d84abfc48d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:10.442070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fb992-1cfb-47a7-8d26-354a7cf28cc1 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:10.449170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bf1b5-473c-4f6a-911e-e134aa8afc0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:10.449757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bf1b5-473c-4f6a-911e-e134aa8afc0f service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.450923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bf1b5-473c-4f6a-911e-e134aa8afc0f service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.450923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bf1b5-473c-4f6a-911e-e134aa8afc0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.549371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17246dc8-fc56-4a4f-b632-d9d84abfc48d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:10.569058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70b5a0-4936-49c0-b280-38fdb3946c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:10.570491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bf1b5-473c-4f6a-911e-e134aa8afc0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:10.572299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bf1b5-473c-4f6a-911e-e134aa8afc0f service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:10.573377 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b5a0-4936-49c0-b280-38fdb3946c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.574071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b5a0-4936-49c0-b280-38fdb3946c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.575272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b5a0-4936-49c0-b280-38fdb3946c5e admin admin] PATH_INFO: `/v3/projects/9b56ea0be2964e21985deb5964232923/users/bf3817e18de149aba43a2f65f37d63f4/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.592848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 808/1612] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:10.594462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e8807-127d-44a4-a66b-7ac2cd295254 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.594862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e8807-127d-44a4-a66b-7ac2cd295254 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.595155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e8807-127d-44a4-a66b-7ac2cd295254 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.597412 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 809/1613] 10.222.0.75 () {54 vars in 912 bytes} [Thu Jul 14 11:07:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:07:10.598630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdf6dd-73b9-4cb3-979a-a4e19b348db2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.598920 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdf6dd-73b9-4cb3-979a-a4e19b348db2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.599284 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdf6dd-73b9-4cb3-979a-a4e19b348db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.621480 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-11fdf6dd-73b9-4cb3-979a-a4e19b348db2 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:10.623760 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-11fdf6dd-73b9-4cb3-979a-a4e19b348db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:10.631514 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70b5a0-4936-49c0-b280-38fdb3946c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b56ea0be2964e21985deb5964232923, user_id=bf3817e18de149aba43a2f65f37d63f4, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:10.637794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70b5a0-4936-49c0-b280-38fdb3946c5e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:10.649007 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 810/1614] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:10.650510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fdf6dd-73b9-4cb3-979a-a4e19b348db2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:10.664636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fdf6dd-73b9-4cb3-979a-a4e19b348db2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:10.674832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 805/1615] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:10 2022] PUT /identity/v3/projects/9b56ea0be2964e21985deb5964232923/users/bf3817e18de149aba43a2f65f37d63f4/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:10.678648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec6531-908a-4960-bb6e-b8f78d7a863a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.680224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec6531-908a-4960-bb6e-b8f78d7a863a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.681281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec6531-908a-4960-bb6e-b8f78d7a863a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.695706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:10.732089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-85ec6531-908a-4960-bb6e-b8f78d7a863a None None] MFA Rules not processed for user `02052e514aaa43d9af166f2bf092d93c`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:10.735399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-85ec6531-908a-4960-bb6e-b8f78d7a863a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:10.754182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b42b6-07fb-44c1-a2aa-e63a87665e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:10.755321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b42b6-07fb-44c1-a2aa-e63a87665e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.755855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b42b6-07fb-44c1-a2aa-e63a87665e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.756150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b42b6-07fb-44c1-a2aa-e63a87665e5f admin admin] PATH_INFO: `/v3/projects/b3f043134a3d4369983f36769d2ae197/users/cb80e0c64c3348cfac5ad2c4f82fb606/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.781391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b42b6-07fb-44c1-a2aa-e63a87665e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f043134a3d4369983f36769d2ae197, user_id=cb80e0c64c3348cfac5ad2c4f82fb606, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:10.785520 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b42b6-07fb-44c1-a2aa-e63a87665e5f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:10.803475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 811/1616] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:10 2022] PUT /identity/v3/projects/b3f043134a3d4369983f36769d2ae197/users/cb80e0c64c3348cfac5ad2c4f82fb606/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:10.808604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b42b6-07fb-44c1-a2aa-e63a87665e5f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:10.839898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 806/1617] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:10.842160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc54cd-7d84-44f0-a4d1-699b482d23c1 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.842774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc54cd-7d84-44f0-a4d1-699b482d23c1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.843282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc54cd-7d84-44f0-a4d1-699b482d23c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.880922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-85cc54cd-7d84-44f0-a4d1-699b482d23c1 None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:10.883135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-85cc54cd-7d84-44f0-a4d1-699b482d23c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:10.909083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b42b6-07fb-44c1-a2aa-e63a87665e5f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:10.917394 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244887a-eaf3-4d4a-a40a-3b58f26286ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:10.918197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244887a-eaf3-4d4a-a40a-3b58f26286ad admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.919065 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244887a-eaf3-4d4a-a40a-3b58f26286ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.919356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244887a-eaf3-4d4a-a40a-3b58f26286ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:10.920179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244887a-eaf3-4d4a-a40a-3b58f26286ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:10.922208 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244887a-eaf3-4d4a-a40a-3b58f26286ad admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:10.954114 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 812/1618] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:10 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:10.955832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244887a-eaf3-4d4a-a40a-3b58f26286ad admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:10.964106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:10.993335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 807/1619] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:07:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:10.995657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd7109-08ca-40be-8b50-34b310961e21 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:10.996221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd7109-08ca-40be-8b50-34b310961e21 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:10.996769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd7109-08ca-40be-8b50-34b310961e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:11.022193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9cfd7109-08ca-40be-8b50-34b310961e21 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:11.023608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9cfd7109-08ca-40be-8b50-34b310961e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:11.053665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244887a-eaf3-4d4a-a40a-3b58f26286ad admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:11.059613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941ba24-01d8-4bb8-9f4b-df0a1a9f80d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:11.060871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941ba24-01d8-4bb8-9f4b-df0a1a9f80d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:11.061371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941ba24-01d8-4bb8-9f4b-df0a1a9f80d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:11.061974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941ba24-01d8-4bb8-9f4b-df0a1a9f80d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:11.120821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 808/1620] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:11.122684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe22c6-9810-44d8-95ea-0c09f3eebbac None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:11.122987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe22c6-9810-44d8-95ea-0c09f3eebbac None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:11.123273 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe22c6-9810-44d8-95ea-0c09f3eebbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:11.144717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-01fe22c6-9810-44d8-95ea-0c09f3eebbac None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:11.145959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-01fe22c6-9810-44d8-95ea-0c09f3eebbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:11.146805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:11.157108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 809/1621] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:11.158795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0705fe-42ba-490d-8a54-d4ac5f00b7a6 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:11.159092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0705fe-42ba-490d-8a54-d4ac5f00b7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:11.159375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0705fe-42ba-490d-8a54-d4ac5f00b7a6 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:11.161055 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 810/1622] 10.222.0.75 () {54 vars in 912 bytes} [Thu Jul 14 11:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:07:11.164290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0705fe-42ba-490d-8a54-d4ac5f00b7a6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:11.180465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941ba24-01d8-4bb8-9f4b-df0a1a9f80d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:11.182508 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941ba24-01d8-4bb8-9f4b-df0a1a9f80d2 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:11.197547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 813/1623] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:11.197547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941ba24-01d8-4bb8-9f4b-df0a1a9f80d2 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:11.206668 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941ba24-01d8-4bb8-9f4b-df0a1a9f80d2 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:11.212493 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5b8a0-d603-4e7b-af7f-65dbad6cea3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:11.213269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5b8a0-d603-4e7b-af7f-65dbad6cea3d admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:11.213570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5b8a0-d603-4e7b-af7f-65dbad6cea3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:11.215726 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5b8a0-d603-4e7b-af7f-65dbad6cea3d admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:11.215726 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5b8a0-d603-4e7b-af7f-65dbad6cea3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:11.217762 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5b8a0-d603-4e7b-af7f-65dbad6cea3d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:11.244645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 814/1624] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:11 2022] POST /identity/v3/users => generated 296 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:11.245842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5b8a0-d603-4e7b-af7f-65dbad6cea3d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:11.281204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0705fe-42ba-490d-8a54-d4ac5f00b7a6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:11.288388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3b01d-2fea-4353-8cc7-e70edafb9d79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:11.289672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3b01d-2fea-4353-8cc7-e70edafb9d79 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:11.290225 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3b01d-2fea-4353-8cc7-e70edafb9d79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:11.290890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3b01d-2fea-4353-8cc7-e70edafb9d79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:11.326384 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5b8a0-d603-4e7b-af7f-65dbad6cea3d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:11.330469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e6ff2-7934-4530-9010-cc5ebec5681c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:11.331146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e6ff2-7934-4530-9010-cc5ebec5681c service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:11.331459 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e6ff2-7934-4530-9010-cc5ebec5681c service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:11.331744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e6ff2-7934-4530-9010-cc5ebec5681c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:11.396211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3b01d-2fea-4353-8cc7-e70edafb9d79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:11.398294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3b01d-2fea-4353-8cc7-e70edafb9d79 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:11.430974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e6ff2-7934-4530-9010-cc5ebec5681c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:11.432894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e6ff2-7934-4530-9010-cc5ebec5681c service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:11.435488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 811/1625] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:11.436652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3b01d-2fea-4353-8cc7-e70edafb9d79 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:11.463127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 815/1626] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:11.468899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da887095-d68c-407f-991c-e65120b53d3b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:11.472077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da887095-d68c-407f-991c-e65120b53d3b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:11.472738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da887095-d68c-407f-991c-e65120b53d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:11.484466 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3b01d-2fea-4353-8cc7-e70edafb9d79 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:11.499266 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa63850-1809-4642-b13e-291e02356642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:11.499907 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa63850-1809-4642-b13e-291e02356642 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:11.500204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa63850-1809-4642-b13e-291e02356642 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:11.500492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa63850-1809-4642-b13e-291e02356642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:11.520562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-da887095-d68c-407f-991c-e65120b53d3b None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:11.523905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-da887095-d68c-407f-991c-e65120b53d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:11.525379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa63850-1809-4642-b13e-291e02356642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:11.554675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 816/1627] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:07:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:11.558355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa63850-1809-4642-b13e-291e02356642 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:11.560735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da887095-d68c-407f-991c-e65120b53d3b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:11.578442 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da887095-d68c-407f-991c-e65120b53d3b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:11.587540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81945a9-1bf5-44c7-a859-7d38d6e2d537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:11.588230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81945a9-1bf5-44c7-a859-7d38d6e2d537 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:11.588521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81945a9-1bf5-44c7-a859-7d38d6e2d537 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:11.588807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81945a9-1bf5-44c7-a859-7d38d6e2d537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:11.610374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 812/1628] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:11.614677 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81945a9-1bf5-44c7-a859-7d38d6e2d537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:11.616437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81945a9-1bf5-44c7-a859-7d38d6e2d537 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:11.617705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa63850-1809-4642-b13e-291e02356642 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:11.645539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 817/1629] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:11.651409 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa63850-1809-4642-b13e-291e02356642 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:11.665582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c699a-e3b1-416d-b461-115bdeaf552a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:11.666384 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c699a-e3b1-416d-b461-115bdeaf552a admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:11.666722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c699a-e3b1-416d-b461-115bdeaf552a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:11.667136 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c699a-e3b1-416d-b461-115bdeaf552a admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:11.668041 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c699a-e3b1-416d-b461-115bdeaf552a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:11.677369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c699a-e3b1-416d-b461-115bdeaf552a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:11.712280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 813/1630] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:11.713283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81945a9-1bf5-44c7-a859-7d38d6e2d537 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:11.727227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81945a9-1bf5-44c7-a859-7d38d6e2d537 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:11.741639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea1990-4ac3-4604-a8ae-94f5328d5dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:11.742576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea1990-4ac3-4604-a8ae-94f5328d5dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:11.744178 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea1990-4ac3-4604-a8ae-94f5328d5dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:11.744483 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea1990-4ac3-4604-a8ae-94f5328d5dcc admin admin] PATH_INFO: `/v3/projects/ab7291487d9840e5bfb396450a5cc98d/users/c56bc04ac4424ba186a07aff7aa9514e/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:11.780461 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea1990-4ac3-4604-a8ae-94f5328d5dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7291487d9840e5bfb396450a5cc98d, user_id=c56bc04ac4424ba186a07aff7aa9514e, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:11.785734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea1990-4ac3-4604-a8ae-94f5328d5dcc admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:11.820923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 818/1631] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:11 2022] PUT /identity/v3/projects/ab7291487d9840e5bfb396450a5cc98d/users/c56bc04ac4424ba186a07aff7aa9514e/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:11.826085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c699a-e3b1-416d-b461-115bdeaf552a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:11.842743 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c699a-e3b1-416d-b461-115bdeaf552a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:11.873026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:12.019233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e7afe-72db-4969-abe2-018f551f55d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:12.019966 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e7afe-72db-4969-abe2-018f551f55d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:12.020258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e7afe-72db-4969-abe2-018f551f55d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:12.020636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e7afe-72db-4969-abe2-018f551f55d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:12.021359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e7afe-72db-4969-abe2-018f551f55d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:12.023190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e7afe-72db-4969-abe2-018f551f55d7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:12.052083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 814/1632] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:12.053604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea1990-4ac3-4604-a8ae-94f5328d5dcc admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:12.071005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea1990-4ac3-4604-a8ae-94f5328d5dcc admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:12.091637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f34eed-387f-4899-bf64-f4f1e3ae1f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:12.092631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f34eed-387f-4899-bf64-f4f1e3ae1f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:12.095227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f34eed-387f-4899-bf64-f4f1e3ae1f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:12.100539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f34eed-387f-4899-bf64-f4f1e3ae1f3e admin admin] PATH_INFO: `/v3/projects/ab7291487d9840e5bfb396450a5cc98d/users/c56bc04ac4424ba186a07aff7aa9514e/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:12.105113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f34eed-387f-4899-bf64-f4f1e3ae1f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7291487d9840e5bfb396450a5cc98d, user_id=c56bc04ac4424ba186a07aff7aa9514e, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:12.116126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f34eed-387f-4899-bf64-f4f1e3ae1f3e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:12.154635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 819/1633] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:12 2022] PUT /identity/v3/projects/ab7291487d9840e5bfb396450a5cc98d/users/c56bc04ac4424ba186a07aff7aa9514e/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:12.166910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa85ec0-3b71-4838-8ce2-6fbda9195464 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:12.167343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa85ec0-3b71-4838-8ce2-6fbda9195464 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:12.167664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa85ec0-3b71-4838-8ce2-6fbda9195464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:12.256170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1aa85ec0-3b71-4838-8ce2-6fbda9195464 None None] MFA Rules not processed for user `c56bc04ac4424ba186a07aff7aa9514e`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:12.261580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1aa85ec0-3b71-4838-8ce2-6fbda9195464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:12.407031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f34eed-387f-4899-bf64-f4f1e3ae1f3e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:12.413140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:12.471815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 815/1634] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:12.577170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f34eed-387f-4899-bf64-f4f1e3ae1f3e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:12.583468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4317f7e-927e-4fa5-92a5-4e8456fb2a59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:12.586470 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4317f7e-927e-4fa5-92a5-4e8456fb2a59 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:12.586856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4317f7e-927e-4fa5-92a5-4e8456fb2a59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:12.587234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4317f7e-927e-4fa5-92a5-4e8456fb2a59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:12.704061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa85ec0-3b71-4838-8ce2-6fbda9195464 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:12.721014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa85ec0-3b71-4838-8ce2-6fbda9195464 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:12.736194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0d1c3-e6d0-45e7-b782-522d2e60d6d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:12.736961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0d1c3-e6d0-45e7-b782-522d2e60d6d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:12.737255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0d1c3-e6d0-45e7-b782-522d2e60d6d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:12.737538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0d1c3-e6d0-45e7-b782-522d2e60d6d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:12.758734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0d1c3-e6d0-45e7-b782-522d2e60d6d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:12.760358 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0d1c3-e6d0-45e7-b782-522d2e60d6d8 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:12.790643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 816/1635] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:12.828722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4317f7e-927e-4fa5-92a5-4e8456fb2a59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:12.831497 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4317f7e-927e-4fa5-92a5-4e8456fb2a59 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:12.909130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 820/1636] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:07:13.751256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0d1c3-e6d0-45e7-b782-522d2e60d6d8 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:13.762533 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0d1c3-e6d0-45e7-b782-522d2e60d6d8 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:13.775600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45419be-d8b0-4340-bf69-242afd13e880 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56bc04ac4424ba186a07aff7aa9514e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab7291487d9840e5bfb396450a5cc98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:13.776652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45419be-d8b0-4340-bf69-242afd13e880 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:13.777277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45419be-d8b0-4340-bf69-242afd13e880 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:13.777581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45419be-d8b0-4340-bf69-242afd13e880 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:13.778617 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45419be-d8b0-4340-bf69-242afd13e880 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:13.779026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45419be-d8b0-4340-bf69-242afd13e880 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:13.781940 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45419be-d8b0-4340-bf69-242afd13e880 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:13.844557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 817/1637] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:13.844985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4317f7e-927e-4fa5-92a5-4e8456fb2a59 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:13.863770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4317f7e-927e-4fa5-92a5-4e8456fb2a59 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:13.873246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9265dd9-284e-4e4c-a107-89db46f9d0eb tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56bc04ac4424ba186a07aff7aa9514e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab7291487d9840e5bfb396450a5cc98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:13.874058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9265dd9-284e-4e4c-a107-89db46f9d0eb tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:13.874768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9265dd9-284e-4e4c-a107-89db46f9d0eb tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:13.875947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9265dd9-284e-4e4c-a107-89db46f9d0eb tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:13.875947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9265dd9-284e-4e4c-a107-89db46f9d0eb tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:13.881233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9265dd9-284e-4e4c-a107-89db46f9d0eb tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:13.987361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45419be-d8b0-4340-bf69-242afd13e880 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:13.993620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 821/1638] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:13 2022] POST /identity/v3/projects => generated 353 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:14.009258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45419be-d8b0-4340-bf69-242afd13e880 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:14.144448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d207e4c-1f92-4b75-a84d-752b23850ae2 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56bc04ac4424ba186a07aff7aa9514e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab7291487d9840e5bfb396450a5cc98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:14.145635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d207e4c-1f92-4b75-a84d-752b23850ae2 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:14.145987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d207e4c-1f92-4b75-a84d-752b23850ae2 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:14.146434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d207e4c-1f92-4b75-a84d-752b23850ae2 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:14.147660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d207e4c-1f92-4b75-a84d-752b23850ae2 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:14.148056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d207e4c-1f92-4b75-a84d-752b23850ae2 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:14.150937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d207e4c-1f92-4b75-a84d-752b23850ae2 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:14.169955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 818/1639] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:07:14.253859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9265dd9-284e-4e4c-a107-89db46f9d0eb tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:14.276372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9265dd9-284e-4e4c-a107-89db46f9d0eb tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:14.285276 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5b9e5-67aa-493d-8096-486375c96bc8 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c56bc04ac4424ba186a07aff7aa9514e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab7291487d9840e5bfb396450a5cc98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:14.286201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5b9e5-67aa-493d-8096-486375c96bc8 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:14.286610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5b9e5-67aa-493d-8096-486375c96bc8 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:14.286881 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5b9e5-67aa-493d-8096-486375c96bc8 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] PATH_INFO: `/v3/projects/de8e6a7ebc9a427ba150f4af75d23dcd` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:14.290787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5b9e5-67aa-493d-8096-486375c96bc8 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: Authorizing `identity:delete_project(project_id=de8e6a7ebc9a427ba150f4af75d23dcd)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:14.295490 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5b9e5-67aa-493d-8096-486375c96bc8 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:14.440566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f7a5b9e5-67aa-493d-8096-486375c96bc8 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de8e6a7ebc9a427ba150f4af75d23dcd', 'request_id': 'req-f7a5b9e5-67aa-493d-8096-486375c96bc8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:14.440566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f7a5b9e5-67aa-493d-8096-486375c96bc8 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de8e6a7ebc9a427ba150f4af75d23dcd', 'request_id': 'req-f7a5b9e5-67aa-493d-8096-486375c96bc8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:14.440566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f7a5b9e5-67aa-493d-8096-486375c96bc8 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de8e6a7ebc9a427ba150f4af75d23dcd', 'request_id': 'req-f7a5b9e5-67aa-493d-8096-486375c96bc8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:14.441987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f7a5b9e5-67aa-493d-8096-486375c96bc8 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] The token cache is being invalidate because project de8e6a7ebc9a427ba150f4af75d23dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:14.441987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f7a5b9e5-67aa-493d-8096-486375c96bc8 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:14.441987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f7a5b9e5-67aa-493d-8096-486375c96bc8 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:14.441987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 822/1640] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:14 2022] DELETE /identity/v3/projects/de8e6a7ebc9a427ba150f4af75d23dcd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:14.828845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d207e4c-1f92-4b75-a84d-752b23850ae2 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:14.831206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5d207e4c-1f92-4b75-a84d-752b23850ae2 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:14.845453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:15.048212 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d207e4c-1f92-4b75-a84d-752b23850ae2 tempest-VolumeQuotaClassesTest-503113366 tempest-VolumeQuotaClassesTest-503113366-project-admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:15.250923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:15.252276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:15.252738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:15.253174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] PATH_INFO: `/v3/users/ea9bcb761c7a4655a35da1948d35ef4c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:15.256652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9bcb761c7a4655a35da1948d35ef4c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:15.260160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:15.338304 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9bcb761c7a4655a35da1948d35ef4c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:15.339102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9bcb761c7a4655a35da1948d35ef4c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:15.358353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9bcb761c7a4655a35da1948d35ef4c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:15.379162 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9bcb761c7a4655a35da1948d35ef4c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:15.386969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9bcb761c7a4655a35da1948d35ef4c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:15.425177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 819/1641] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:14 2022] DELETE /identity/v3/users/ea9bcb761c7a4655a35da1948d35ef4c => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:15.431139 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47ff30-7a4f-488d-912f-ce6d8f0cfc0f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:15.431790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47ff30-7a4f-488d-912f-ce6d8f0cfc0f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:15.432057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47ff30-7a4f-488d-912f-ce6d8f0cfc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:15.453911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:15.468095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bd47ff30-7a4f-488d-912f-ce6d8f0cfc0f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:15.473942 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bd47ff30-7a4f-488d-912f-ce6d8f0cfc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:15.586855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 823/1642] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:15.661819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:15.669769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:15.805481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47ff30-7a4f-488d-912f-ce6d8f0cfc0f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:15.808275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bd47ff30-7a4f-488d-912f-ce6d8f0cfc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:15.837618 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:15.845669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47ff30-7a4f-488d-912f-ce6d8f0cfc0f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:15.861669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:15.867070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:15.867365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:15.867694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] PATH_INFO: `/v3/users/e4b3fb9e24a24db7afb30583737f1314` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:15.874108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4b3fb9e24a24db7afb30583737f1314)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:15.879853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: WARNING py.warnings [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:07:15.879853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: warnings.warn(msg) Jul 14 11:07:15.879853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: Jul 14 11:07:15.882573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:15.893638 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56ee4c-bd37-4df0-acd2-17eded565d37 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:15.920621 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2aa0f3-965a-4903-89bd-25201fe6722a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:15.921742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aa0f3-965a-4903-89bd-25201fe6722a service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:15.922281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aa0f3-965a-4903-89bd-25201fe6722a service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:15.922908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aa0f3-965a-4903-89bd-25201fe6722a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:15.949619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2aa0f3-965a-4903-89bd-25201fe6722a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:15.953539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2aa0f3-965a-4903-89bd-25201fe6722a service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:15.980357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 820/1643] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:16.085095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4b3fb9e24a24db7afb30583737f1314'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:16.097790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4b3fb9e24a24db7afb30583737f1314'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:16.098979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4b3fb9e24a24db7afb30583737f1314'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:16.133280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4b3fb9e24a24db7afb30583737f1314'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:16.135837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4b3fb9e24a24db7afb30583737f1314'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:16.239028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 824/1644] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:15 2022] DELETE /identity/v3/users/e4b3fb9e24a24db7afb30583737f1314 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:16.240363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6ef33-d834-4b00-b7f0-c80c5e0ddea6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:16.240896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6ef33-d834-4b00-b7f0-c80c5e0ddea6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:16.241374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6ef33-d834-4b00-b7f0-c80c5e0ddea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:16.293919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f5b6ef33-d834-4b00-b7f0-c80c5e0ddea6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:16.295539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f5b6ef33-d834-4b00-b7f0-c80c5e0ddea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:16.453371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 821/1645] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:16.502471 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:16.504757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:16.695238 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640696ae-8eab-4c7d-a362-9dbdcfa71fb1 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:16.704529 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9bfcb-a47b-44f5-96ae-44bef1549732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:16.705041 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9bfcb-a47b-44f5-96ae-44bef1549732 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:16.705400 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9bfcb-a47b-44f5-96ae-44bef1549732 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:16.705754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9bfcb-a47b-44f5-96ae-44bef1549732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:16.720833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9bfcb-a47b-44f5-96ae-44bef1549732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:16.723415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9bfcb-a47b-44f5-96ae-44bef1549732 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:16.737860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 825/1646] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:17.522847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6ef33-d834-4b00-b7f0-c80c5e0ddea6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:17.524384 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f5b6ef33-d834-4b00-b7f0-c80c5e0ddea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:17.550495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6ef33-d834-4b00-b7f0-c80c5e0ddea6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:17.563283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:17.564419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:17.564800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:17.565198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] PATH_INFO: `/v3/projects/02935d215ec341e189241e3a6c9e2747` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:17.568050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02935d215ec341e189241e3a6c9e2747)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:17.569242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: WARNING py.warnings [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 11:07:17.569242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: warnings.warn(msg) Jul 14 11:07:17.569242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: Jul 14 11:07:17.570696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:17.687062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02935d215ec341e189241e3a6c9e2747', 'request_id': 'req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:17.706320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02935d215ec341e189241e3a6c9e2747', 'request_id': 'req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:17.717248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02935d215ec341e189241e3a6c9e2747', 'request_id': 'req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:17.755732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] The token cache is being invalidate because project 02935d215ec341e189241e3a6c9e2747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:17.756097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:17.757460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:17.763882 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 822/1647] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:17 2022] DELETE /identity/v3/projects/02935d215ec341e189241e3a6c9e2747 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:17.765160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9bfcb-a47b-44f5-96ae-44bef1549732 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:17.779645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9bfcb-a47b-44f5-96ae-44bef1549732 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:17.825240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:17.946805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147af03f-53ef-4411-9e56-bc4365317ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:17.947783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147af03f-53ef-4411-9e56-bc4365317ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:17.948074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147af03f-53ef-4411-9e56-bc4365317ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:17.948357 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147af03f-53ef-4411-9e56-bc4365317ae3 admin admin] PATH_INFO: `/v3/users/c56bc04ac4424ba186a07aff7aa9514e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:17.951155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147af03f-53ef-4411-9e56-bc4365317ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c56bc04ac4424ba186a07aff7aa9514e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:17.960010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147af03f-53ef-4411-9e56-bc4365317ae3 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:18.075895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-147af03f-53ef-4411-9e56-bc4365317ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c56bc04ac4424ba186a07aff7aa9514e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:18.095149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-147af03f-53ef-4411-9e56-bc4365317ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c56bc04ac4424ba186a07aff7aa9514e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:18.097653 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-147af03f-53ef-4411-9e56-bc4365317ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c56bc04ac4424ba186a07aff7aa9514e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:18.135983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-147af03f-53ef-4411-9e56-bc4365317ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c56bc04ac4424ba186a07aff7aa9514e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:18.138182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-147af03f-53ef-4411-9e56-bc4365317ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c56bc04ac4424ba186a07aff7aa9514e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:18.208166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 826/1648] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:17 2022] DELETE /identity/v3/users/c56bc04ac4424ba186a07aff7aa9514e => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:19.007929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:19.009863 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:19.040116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cb47d-addd-4ab4-bd54-d925bbfcbad8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:19.170184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:19.184125 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795f081-08dc-4ffd-85a1-0e85a0be4899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:19.185260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795f081-08dc-4ffd-85a1-0e85a0be4899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:19.195514 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795f081-08dc-4ffd-85a1-0e85a0be4899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:19.196093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795f081-08dc-4ffd-85a1-0e85a0be4899 admin admin] PATH_INFO: `/v3/projects/e4a8af112fa744a185f2dfe34224f554` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:19.197785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795f081-08dc-4ffd-85a1-0e85a0be4899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4a8af112fa744a185f2dfe34224f554)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:19.201593 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795f081-08dc-4ffd-85a1-0e85a0be4899 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:19.295982 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8795f081-08dc-4ffd-85a1-0e85a0be4899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4a8af112fa744a185f2dfe34224f554', 'request_id': 'req-8795f081-08dc-4ffd-85a1-0e85a0be4899'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:19.313987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8795f081-08dc-4ffd-85a1-0e85a0be4899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4a8af112fa744a185f2dfe34224f554', 'request_id': 'req-8795f081-08dc-4ffd-85a1-0e85a0be4899'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:19.336705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8795f081-08dc-4ffd-85a1-0e85a0be4899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4a8af112fa744a185f2dfe34224f554', 'request_id': 'req-8795f081-08dc-4ffd-85a1-0e85a0be4899'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:19.375618 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8795f081-08dc-4ffd-85a1-0e85a0be4899 admin admin] The token cache is being invalidate because project e4a8af112fa744a185f2dfe34224f554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:19.376016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8795f081-08dc-4ffd-85a1-0e85a0be4899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:19.377624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8795f081-08dc-4ffd-85a1-0e85a0be4899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:19.386748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 823/1649] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:19 2022] DELETE /identity/v3/projects/e4a8af112fa744a185f2dfe34224f554 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:19.387808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f4953-ece2-4089-90e6-1ab0b37fc8a8 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:19.393450 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f4953-ece2-4089-90e6-1ab0b37fc8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:19.394156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f4953-ece2-4089-90e6-1ab0b37fc8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:19.448009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-569f4953-ece2-4089-90e6-1ab0b37fc8a8 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:19.450921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-569f4953-ece2-4089-90e6-1ab0b37fc8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:19.708145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 827/1650] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:19.715616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef42702-30a8-4b5e-b02e-938da9eff672 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:19.715917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef42702-30a8-4b5e-b02e-938da9eff672 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:19.716227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef42702-30a8-4b5e-b02e-938da9eff672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:19.745320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5ef42702-30a8-4b5e-b02e-938da9eff672 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:19.748802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5ef42702-30a8-4b5e-b02e-938da9eff672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:19.778619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f4953-ece2-4089-90e6-1ab0b37fc8a8 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:19.788874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 824/1651] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:19.794951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f4953-ece2-4089-90e6-1ab0b37fc8a8 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:19.800466 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25d886-0c12-408b-91b5-811f879759f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:19.800942 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25d886-0c12-408b-91b5-811f879759f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:19.801235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25d886-0c12-408b-91b5-811f879759f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:19.801517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25d886-0c12-408b-91b5-811f879759f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:19.802436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25d886-0c12-408b-91b5-811f879759f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:19.803343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25d886-0c12-408b-91b5-811f879759f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:19.809606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25d886-0c12-408b-91b5-811f879759f8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:19.828559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 828/1652] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:19.828766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef42702-30a8-4b5e-b02e-938da9eff672 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:19.865983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef42702-30a8-4b5e-b02e-938da9eff672 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:19.882952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68403b-36e8-47e0-9468-5bb39c15e3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:19.884968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68403b-36e8-47e0-9468-5bb39c15e3b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:19.886288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68403b-36e8-47e0-9468-5bb39c15e3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:19.887315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68403b-36e8-47e0-9468-5bb39c15e3b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:19.892280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68403b-36e8-47e0-9468-5bb39c15e3b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:19.899370 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68403b-36e8-47e0-9468-5bb39c15e3b0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:19.960414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25d886-0c12-408b-91b5-811f879759f8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:19.962983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0d25d886-0c12-408b-91b5-811f879759f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:19.984448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 825/1653] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:19 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:07:19.985521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68403b-36e8-47e0-9468-5bb39c15e3b0 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:20.011031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25d886-0c12-408b-91b5-811f879759f8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:20.077068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:20.146768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:20.148118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:20.148557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:20.149032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465 admin admin] PATH_INFO: `/v3/projects/ab7291487d9840e5bfb396450a5cc98d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:20.154386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7291487d9840e5bfb396450a5cc98d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:20.156822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:20.176285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68403b-36e8-47e0-9468-5bb39c15e3b0 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:20.189963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76138a92-022a-4f91-a0a3-e03ad044585a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:20.191460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76138a92-022a-4f91-a0a3-e03ad044585a admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:20.191983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76138a92-022a-4f91-a0a3-e03ad044585a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:20.192466 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76138a92-022a-4f91-a0a3-e03ad044585a admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:20.193388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76138a92-022a-4f91-a0a3-e03ad044585a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:20.204543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76138a92-022a-4f91-a0a3-e03ad044585a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:20.241955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7291487d9840e5bfb396450a5cc98d', 'request_id': 'req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:20.254083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7291487d9840e5bfb396450a5cc98d', 'request_id': 'req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:20.263192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7291487d9840e5bfb396450a5cc98d', 'request_id': 'req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:20.269137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 826/1654] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:19 2022] POST /identity/v3/users => generated 287 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:20.270730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76138a92-022a-4f91-a0a3-e03ad044585a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:20.293106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465 admin admin] The token cache is being invalidate because project ab7291487d9840e5bfb396450a5cc98d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:20.293455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:20.294600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76138a92-022a-4f91-a0a3-e03ad044585a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:20.295085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-bc80d7cc-fd82-4a39-ab77-d3c5eb582465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:20.308893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 829/1655] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:19 2022] DELETE /identity/v3/projects/ab7291487d9840e5bfb396450a5cc98d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:20.310905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5ea5e-8dac-4f64-ab09-cec9a50e3768 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:20.311262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5ea5e-8dac-4f64-ab09-cec9a50e3768 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:20.311619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5ea5e-8dac-4f64-ab09-cec9a50e3768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:20.368838 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2cc5ea5e-8dac-4f64-ab09-cec9a50e3768 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:20.373200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2cc5ea5e-8dac-4f64-ab09-cec9a50e3768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:20.411070 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:20.438987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ac94f-7f07-41c0-90b1-79660c61aff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:20.439737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ac94f-7f07-41c0-90b1-79660c61aff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:20.440024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ac94f-7f07-41c0-90b1-79660c61aff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:20.440298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ac94f-7f07-41c0-90b1-79660c61aff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:20.441058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18ac94f-7f07-41c0-90b1-79660c61aff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:20.442702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18ac94f-7f07-41c0-90b1-79660c61aff5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:20.490623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 827/1656] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:20.492726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ac94f-7f07-41c0-90b1-79660c61aff5 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:20.497484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a18ac94f-7f07-41c0-90b1-79660c61aff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:20.558311 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ac94f-7f07-41c0-90b1-79660c61aff5 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:20.574031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 830/1657] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:20.576047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff19c4-6981-4c26-a4dc-cc479462a23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:20.578147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff19c4-6981-4c26-a4dc-cc479462a23a admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:20.578468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff19c4-6981-4c26-a4dc-cc479462a23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:20.578731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff19c4-6981-4c26-a4dc-cc479462a23a admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:20.579662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff19c4-6981-4c26-a4dc-cc479462a23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:20.585007 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296b242-5690-4fc5-85ad-3e0cce7d8f6b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:20.585491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296b242-5690-4fc5-85ad-3e0cce7d8f6b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:20.586042 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296b242-5690-4fc5-85ad-3e0cce7d8f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:20.591853 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff19c4-6981-4c26-a4dc-cc479462a23a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:20.620183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 828/1658] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:20.628216 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff19c4-6981-4c26-a4dc-cc479462a23a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:20.641966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2296b242-5690-4fc5-85ad-3e0cce7d8f6b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:20.646726 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2296b242-5690-4fc5-85ad-3e0cce7d8f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:20.655577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff19c4-6981-4c26-a4dc-cc479462a23a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:20.673370 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 831/1659] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:20.673949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c54fb2-b283-4e9e-9990-0ea0bd252718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:20.675741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296b242-5690-4fc5-85ad-3e0cce7d8f6b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:20.681237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c54fb2-b283-4e9e-9990-0ea0bd252718 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:20.681605 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c54fb2-b283-4e9e-9990-0ea0bd252718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:20.681898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c54fb2-b283-4e9e-9990-0ea0bd252718 admin admin] PATH_INFO: `/v3/projects/8362e5675331424982c4e018bbfea642/users/02ee4a7254e9460f8169c412de8bbcb5/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:20.717208 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296b242-5690-4fc5-85ad-3e0cce7d8f6b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:20.729759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa3b08-8561-41ae-85bb-c7634af129f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:20.730964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa3b08-8561-41ae-85bb-c7634af129f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:20.732327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa3b08-8561-41ae-85bb-c7634af129f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:20.733618 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa3b08-8561-41ae-85bb-c7634af129f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:20.734569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa3b08-8561-41ae-85bb-c7634af129f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:20.735284 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa3b08-8561-41ae-85bb-c7634af129f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:20.739993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c54fb2-b283-4e9e-9990-0ea0bd252718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8362e5675331424982c4e018bbfea642, user_id=02ee4a7254e9460f8169c412de8bbcb5, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:20.741665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa3b08-8561-41ae-85bb-c7634af129f9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:20.744479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c54fb2-b283-4e9e-9990-0ea0bd252718 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:20.777353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 832/1660] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:20.780779 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa3b08-8561-41ae-85bb-c7634af129f9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:20.784911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 829/1661] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:20 2022] PUT /identity/v3/projects/8362e5675331424982c4e018bbfea642/users/02ee4a7254e9460f8169c412de8bbcb5/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:20.798945 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d8d0-a02a-4abe-8c5f-6ac3f10f253b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:20.800298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d8d0-a02a-4abe-8c5f-6ac3f10f253b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:20.800670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d8d0-a02a-4abe-8c5f-6ac3f10f253b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:20.814339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa3b08-8561-41ae-85bb-c7634af129f9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:20.874938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3cf4d8d0-a02a-4abe-8c5f-6ac3f10f253b None None] MFA Rules not processed for user `02ee4a7254e9460f8169c412de8bbcb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:20.879246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3cf4d8d0-a02a-4abe-8c5f-6ac3f10f253b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:20.919512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:20.973735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa717d3-f205-4bca-b6d1-70b2de92010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:20.973735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa717d3-f205-4bca-b6d1-70b2de92010c admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:20.973965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa717d3-f205-4bca-b6d1-70b2de92010c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:20.974234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa717d3-f205-4bca-b6d1-70b2de92010c admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:20.975646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa717d3-f205-4bca-b6d1-70b2de92010c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:20.984637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa717d3-f205-4bca-b6d1-70b2de92010c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:21.070175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 833/1662] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:20 2022] POST /identity/v3/projects => generated 385 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:21.075203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa717d3-f205-4bca-b6d1-70b2de92010c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:21.088844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 830/1663] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:21.102992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4d8d0-a02a-4abe-8c5f-6ac3f10f253b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:21.109647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3cf4d8d0-a02a-4abe-8c5f-6ac3f10f253b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:21.291050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa717d3-f205-4bca-b6d1-70b2de92010c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:21.315962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4db7c-ae2b-4c4e-9adc-0410e09a66c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:21.320449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4d8d0-a02a-4abe-8c5f-6ac3f10f253b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:21.320945 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4db7c-ae2b-4c4e-9adc-0410e09a66c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:21.321573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4db7c-ae2b-4c4e-9adc-0410e09a66c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:21.324115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4db7c-ae2b-4c4e-9adc-0410e09a66c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:21.327280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4db7c-ae2b-4c4e-9adc-0410e09a66c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:21.336274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4db7c-ae2b-4c4e-9adc-0410e09a66c8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:21.390544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 834/1664] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:21 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:07:21.397375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4db7c-ae2b-4c4e-9adc-0410e09a66c8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:21.420079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4db7c-ae2b-4c4e-9adc-0410e09a66c8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:21.430332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:21.437235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c166c43-4178-467f-9b6a-be807e5fa5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:21.439987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c166c43-4178-467f-9b6a-be807e5fa5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:21.441774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c166c43-4178-467f-9b6a-be807e5fa5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:21.442150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c166c43-4178-467f-9b6a-be807e5fa5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:21.443140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c166c43-4178-467f-9b6a-be807e5fa5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:21.446521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c166c43-4178-467f-9b6a-be807e5fa5f5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:21.486809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 835/1665] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:21.490897 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c166c43-4178-467f-9b6a-be807e5fa5f5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:21.526456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c166c43-4178-467f-9b6a-be807e5fa5f5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:21.542676 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988c4b5-4611-4419-b620-78b5a7c0ee95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:21.545781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff948e-4142-4517-97ab-33dd27deea16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:21.546555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988c4b5-4611-4419-b620-78b5a7c0ee95 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:21.547645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988c4b5-4611-4419-b620-78b5a7c0ee95 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:21.548712 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff948e-4142-4517-97ab-33dd27deea16 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:21.549492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988c4b5-4611-4419-b620-78b5a7c0ee95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:21.552427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff948e-4142-4517-97ab-33dd27deea16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:21.552576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff948e-4142-4517-97ab-33dd27deea16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:21.556002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff948e-4142-4517-97ab-33dd27deea16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:21.567238 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff948e-4142-4517-97ab-33dd27deea16 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:21.615473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 836/1666] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:21.617536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff948e-4142-4517-97ab-33dd27deea16 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:21.651453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff948e-4142-4517-97ab-33dd27deea16 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:21.665951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee36b7-3705-4246-896c-5463fbc1539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:21.667012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee36b7-3705-4246-896c-5463fbc1539d admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:21.668736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee36b7-3705-4246-896c-5463fbc1539d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:21.669133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee36b7-3705-4246-896c-5463fbc1539d admin admin] PATH_INFO: `/v3/projects/fa15e7e39a1746198bdc2be02bb5eee4/users/b6575d95c20846ab8c945cfb9515e5e8/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:21.710234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee36b7-3705-4246-896c-5463fbc1539d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa15e7e39a1746198bdc2be02bb5eee4, user_id=b6575d95c20846ab8c945cfb9515e5e8, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:21.717224 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee36b7-3705-4246-896c-5463fbc1539d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:21.741415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988c4b5-4611-4419-b620-78b5a7c0ee95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:21.743413 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988c4b5-4611-4419-b620-78b5a7c0ee95 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:21.747595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 837/1667] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:21 2022] PUT /identity/v3/projects/fa15e7e39a1746198bdc2be02bb5eee4/users/b6575d95c20846ab8c945cfb9515e5e8/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:21.770237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb67a86-bb6f-4576-a887-548c2779e7c8 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:21.770722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb67a86-bb6f-4576-a887-548c2779e7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:21.771112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb67a86-bb6f-4576-a887-548c2779e7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:21.775671 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 831/1668] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:21.836638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-1fb67a86-bb6f-4576-a887-548c2779e7c8 None None] MFA Rules not processed for user `b6575d95c20846ab8c945cfb9515e5e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:21.837567 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1fb67a86-bb6f-4576-a887-548c2779e7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:22.052029 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:22.073278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 838/1669] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:22.078658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988c4b5-4611-4419-b620-78b5a7c0ee95 service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:22.108738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988c4b5-4611-4419-b620-78b5a7c0ee95 service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:22.159114 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb67a86-bb6f-4576-a887-548c2779e7c8 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:22.284982 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a0c29-8742-4415-bf04-fd71293472b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:22.285673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a0c29-8742-4415-bf04-fd71293472b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:22.285963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a0c29-8742-4415-bf04-fd71293472b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:22.286247 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a0c29-8742-4415-bf04-fd71293472b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:22.288185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a0c29-8742-4415-bf04-fd71293472b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:22.292286 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a0c29-8742-4415-bf04-fd71293472b8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:22.311667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb67a86-bb6f-4576-a887-548c2779e7c8 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:22.319636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bb31e-b13a-4306-a7db-d77e7176c877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:22.320279 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bb31e-b13a-4306-a7db-d77e7176c877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:22.322739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bb31e-b13a-4306-a7db-d77e7176c877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:22.323060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bb31e-b13a-4306-a7db-d77e7176c877 admin admin] PATH_INFO: `/v3/users/02ee4a7254e9460f8169c412de8bbcb5` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:22.326846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bb31e-b13a-4306-a7db-d77e7176c877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02ee4a7254e9460f8169c412de8bbcb5)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:22.330358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bb31e-b13a-4306-a7db-d77e7176c877 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:22.352929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 832/1670] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:22 2022] POST /identity/v3/projects => generated 383 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:22.358481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a0c29-8742-4415-bf04-fd71293472b8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:22.443522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-846bb31e-b13a-4306-a7db-d77e7176c877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02ee4a7254e9460f8169c412de8bbcb5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:22.466550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-846bb31e-b13a-4306-a7db-d77e7176c877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02ee4a7254e9460f8169c412de8bbcb5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:22.470391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-846bb31e-b13a-4306-a7db-d77e7176c877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02ee4a7254e9460f8169c412de8bbcb5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:22.487743 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-846bb31e-b13a-4306-a7db-d77e7176c877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02ee4a7254e9460f8169c412de8bbcb5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:22.489983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-846bb31e-b13a-4306-a7db-d77e7176c877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02ee4a7254e9460f8169c412de8bbcb5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:22.542064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 839/1671] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:22 2022] DELETE /identity/v3/users/02ee4a7254e9460f8169c412de8bbcb5 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:07:22.547961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd20f4-e7bc-47f9-99a5-18b6dcf3d0e8 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:22.549685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd20f4-e7bc-47f9-99a5-18b6dcf3d0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:22.550141 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd20f4-e7bc-47f9-99a5-18b6dcf3d0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:22.556130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:22.564470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a0c29-8742-4415-bf04-fd71293472b8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:22.585425 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-49cd20f4-e7bc-47f9-99a5-18b6dcf3d0e8 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:22.587744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-49cd20f4-e7bc-47f9-99a5-18b6dcf3d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:22.751282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 840/1672] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:22.762346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befea940-0129-4f8e-afcf-c27859e3347a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:22.763408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befea940-0129-4f8e-afcf-c27859e3347a admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:22.763896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befea940-0129-4f8e-afcf-c27859e3347a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:22.764388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befea940-0129-4f8e-afcf-c27859e3347a admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:22.765309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befea940-0129-4f8e-afcf-c27859e3347a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:22.767579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befea940-0129-4f8e-afcf-c27859e3347a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:22.810970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd20f4-e7bc-47f9-99a5-18b6dcf3d0e8 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:22.812690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-49cd20f4-e7bc-47f9-99a5-18b6dcf3d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:22.833017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 833/1673] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:22 2022] POST /identity/v3/users => generated 292 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:22.834651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befea940-0129-4f8e-afcf-c27859e3347a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:22.845955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-befea940-0129-4f8e-afcf-c27859e3347a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:22.877261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:22.878034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befea940-0129-4f8e-afcf-c27859e3347a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:22.892354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159b0d8-0fe6-420a-98f6-6856b84ceca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:22.893458 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159b0d8-0fe6-420a-98f6-6856b84ceca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:22.893987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159b0d8-0fe6-420a-98f6-6856b84ceca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:22.894480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159b0d8-0fe6-420a-98f6-6856b84ceca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:22.896111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159b0d8-0fe6-420a-98f6-6856b84ceca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:22.898000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159b0d8-0fe6-420a-98f6-6856b84ceca0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:22.925861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 834/1674] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:22.927139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159b0d8-0fe6-420a-98f6-6856b84ceca0 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:22.950158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159b0d8-0fe6-420a-98f6-6856b84ceca0 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:22.963856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99c788-1e43-41ed-976c-fe3481a36aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:22.965586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99c788-1e43-41ed-976c-fe3481a36aff admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:22.966093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99c788-1e43-41ed-976c-fe3481a36aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:22.967533 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99c788-1e43-41ed-976c-fe3481a36aff admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:22.968604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99c788-1e43-41ed-976c-fe3481a36aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:22.972121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99c788-1e43-41ed-976c-fe3481a36aff admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:22.999590 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd20f4-e7bc-47f9-99a5-18b6dcf3d0e8 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:23.003921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 835/1675] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:23.007267 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99c788-1e43-41ed-976c-fe3481a36aff admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:23.013080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c538f-31fb-44ff-a38d-65199a3f6c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:23.017998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c538f-31fb-44ff-a38d-65199a3f6c41 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:23.018298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c538f-31fb-44ff-a38d-65199a3f6c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:23.022933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c538f-31fb-44ff-a38d-65199a3f6c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:23.032678 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99c788-1e43-41ed-976c-fe3481a36aff admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:23.039311 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac6e28-0986-4532-8f80-fc1a31249c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:23.041202 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac6e28-0986-4532-8f80-fc1a31249c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:23.041960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac6e28-0986-4532-8f80-fc1a31249c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:23.044627 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac6e28-0986-4532-8f80-fc1a31249c65 admin admin] PATH_INFO: `/v3/projects/f380151590e645328c138113e7d13081/users/e9b44b5c38294d4f81c849fd8a6aa708/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:23.063297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c538f-31fb-44ff-a38d-65199a3f6c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:23.069234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c538f-31fb-44ff-a38d-65199a3f6c41 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:23.082224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac6e28-0986-4532-8f80-fc1a31249c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f380151590e645328c138113e7d13081, user_id=e9b44b5c38294d4f81c849fd8a6aa708, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:23.090344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac6e28-0986-4532-8f80-fc1a31249c65 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:23.107559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 841/1676] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:07:23.129169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 836/1677] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:23 2022] PUT /identity/v3/projects/f380151590e645328c138113e7d13081/users/e9b44b5c38294d4f81c849fd8a6aa708/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:23.130800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91fe7a-ad4d-41e9-8b42-f068c27d2984 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:23.131096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91fe7a-ad4d-41e9-8b42-f068c27d2984 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:23.131404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91fe7a-ad4d-41e9-8b42-f068c27d2984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:23.202915 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-1e91fe7a-ad4d-41e9-8b42-f068c27d2984 None None] MFA Rules not processed for user `e9b44b5c38294d4f81c849fd8a6aa708`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:23.204376 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1e91fe7a-ad4d-41e9-8b42-f068c27d2984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:23.342921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 842/1678] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:23.347587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac6e28-0986-4532-8f80-fc1a31249c65 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:23.508379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac6e28-0986-4532-8f80-fc1a31249c65 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:23.520601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0bbbb-bf76-4666-a2c4-101c5aa42192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:23.522492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0bbbb-bf76-4666-a2c4-101c5aa42192 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:23.522621 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0bbbb-bf76-4666-a2c4-101c5aa42192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:23.525510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0bbbb-bf76-4666-a2c4-101c5aa42192 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:23.526094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0bbbb-bf76-4666-a2c4-101c5aa42192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:23.537759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0bbbb-bf76-4666-a2c4-101c5aa42192 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:23.583018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 837/1679] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:23 2022] POST /identity/v3/projects => generated 385 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:23.587606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91fe7a-ad4d-41e9-8b42-f068c27d2984 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:23.603962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91fe7a-ad4d-41e9-8b42-f068c27d2984 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:23.749146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267d045-0f29-4bb6-9e54-a53be41f47eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:23.749751 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267d045-0f29-4bb6-9e54-a53be41f47eb admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:23.750076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267d045-0f29-4bb6-9e54-a53be41f47eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:23.750310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267d045-0f29-4bb6-9e54-a53be41f47eb admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:23.751050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9267d045-0f29-4bb6-9e54-a53be41f47eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:23.753078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9267d045-0f29-4bb6-9e54-a53be41f47eb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:23.815954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 843/1680] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:23 2022] POST /identity/v3/users => generated 299 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:23.817373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0bbbb-bf76-4666-a2c4-101c5aa42192 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:23.838954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0bbbb-bf76-4666-a2c4-101c5aa42192 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:23.855814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02122f-bde2-44bb-9b39-a79a66e0da8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:23.857828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02122f-bde2-44bb-9b39-a79a66e0da8a admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:23.858381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02122f-bde2-44bb-9b39-a79a66e0da8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:23.859570 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02122f-bde2-44bb-9b39-a79a66e0da8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:23.860846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02122f-bde2-44bb-9b39-a79a66e0da8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:23.863359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02122f-bde2-44bb-9b39-a79a66e0da8a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:23.909042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 838/1681] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:23.909877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267d045-0f29-4bb6-9e54-a53be41f47eb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:23.925919 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267d045-0f29-4bb6-9e54-a53be41f47eb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:23.942372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be302e-c83e-4ca0-8700-2b487815e38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:23.943825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be302e-c83e-4ca0-8700-2b487815e38f admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:23.944326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be302e-c83e-4ca0-8700-2b487815e38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:23.944709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be302e-c83e-4ca0-8700-2b487815e38f admin admin] PATH_INFO: `/v3/projects/3947ee4dc32e4d7c85b62110269c71f2/users/d3eee652f5c64fd4bf7e7c0d3af1a88e/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:23.996905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be302e-c83e-4ca0-8700-2b487815e38f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3947ee4dc32e4d7c85b62110269c71f2, user_id=d3eee652f5c64fd4bf7e7c0d3af1a88e, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:23.999708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be302e-c83e-4ca0-8700-2b487815e38f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:24.036800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 844/1682] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:23 2022] PUT /identity/v3/projects/3947ee4dc32e4d7c85b62110269c71f2/users/d3eee652f5c64fd4bf7e7c0d3af1a88e/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:07:24.045789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02122f-bde2-44bb-9b39-a79a66e0da8a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:24.077717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02122f-bde2-44bb-9b39-a79a66e0da8a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:24.120422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:24.219676 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f31d27-4f95-4852-b2ca-8295c22e570b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:24.220798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f31d27-4f95-4852-b2ca-8295c22e570b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:24.221483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f31d27-4f95-4852-b2ca-8295c22e570b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:24.222797 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f31d27-4f95-4852-b2ca-8295c22e570b admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:24.224317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f31d27-4f95-4852-b2ca-8295c22e570b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:24.226213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f31d27-4f95-4852-b2ca-8295c22e570b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:24.266750 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 839/1683] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:24.266973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be302e-c83e-4ca0-8700-2b487815e38f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:24.285153 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be302e-c83e-4ca0-8700-2b487815e38f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:24.295310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd169d-d1c3-4c72-a0ae-a4302b9c064a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:24.297824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd169d-d1c3-4c72-a0ae-a4302b9c064a admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:24.300206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd169d-d1c3-4c72-a0ae-a4302b9c064a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:24.300995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd169d-d1c3-4c72-a0ae-a4302b9c064a admin admin] PATH_INFO: `/v3/projects/3947ee4dc32e4d7c85b62110269c71f2/users/d3eee652f5c64fd4bf7e7c0d3af1a88e/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:24.305686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd169d-d1c3-4c72-a0ae-a4302b9c064a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3947ee4dc32e4d7c85b62110269c71f2, user_id=d3eee652f5c64fd4bf7e7c0d3af1a88e, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:24.315059 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd169d-d1c3-4c72-a0ae-a4302b9c064a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:24.359488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 845/1684] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:24 2022] PUT /identity/v3/projects/3947ee4dc32e4d7c85b62110269c71f2/users/d3eee652f5c64fd4bf7e7c0d3af1a88e/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:24.363886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d88a33-564f-475c-8699-564a0b8f57a4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:24.364177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d88a33-564f-475c-8699-564a0b8f57a4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:24.364471 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d88a33-564f-475c-8699-564a0b8f57a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:24.390614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f518c12-b2da-4d19-82c1-bcd96d8982b4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:24.390970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f518c12-b2da-4d19-82c1-bcd96d8982b4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:24.391317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f518c12-b2da-4d19-82c1-bcd96d8982b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:24.398587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:24.446141 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-1f518c12-b2da-4d19-82c1-bcd96d8982b4 None None] MFA Rules not processed for user `fffbf852ddc9455f8a68f80ad118e4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:24.448337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1f518c12-b2da-4d19-82c1-bcd96d8982b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:24.483268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b5d88a33-564f-475c-8699-564a0b8f57a4 None None] MFA Rules not processed for user `d3eee652f5c64fd4bf7e7c0d3af1a88e`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:24.484570 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b5d88a33-564f-475c-8699-564a0b8f57a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:24.673487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 846/1685] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:24.688613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f518c12-b2da-4d19-82c1-bcd96d8982b4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:24.690067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1f518c12-b2da-4d19-82c1-bcd96d8982b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:24.699356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 840/1686] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:24.707302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d88a33-564f-475c-8699-564a0b8f57a4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:24.709932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b5d88a33-564f-475c-8699-564a0b8f57a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:24.844439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d88a33-564f-475c-8699-564a0b8f57a4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:24.857915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d279c-e32d-439d-b031-9075a87ab054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:24.858594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d279c-e32d-439d-b031-9075a87ab054 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:24.858883 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d279c-e32d-439d-b031-9075a87ab054 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:24.859164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d279c-e32d-439d-b031-9075a87ab054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:24.899020 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d279c-e32d-439d-b031-9075a87ab054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:24.900527 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d279c-e32d-439d-b031-9075a87ab054 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:24.906576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f518c12-b2da-4d19-82c1-bcd96d8982b4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:24.921120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30731b75-2387-4f24-a2a4-88e7c0fc5667 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:24.922035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30731b75-2387-4f24-a2a4-88e7c0fc5667 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:24.922322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30731b75-2387-4f24-a2a4-88e7c0fc5667 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:24.922593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30731b75-2387-4f24-a2a4-88e7c0fc5667 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:24.930489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 841/1687] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:07:24.941544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30731b75-2387-4f24-a2a4-88e7c0fc5667 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:24.943181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30731b75-2387-4f24-a2a4-88e7c0fc5667 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:24.954518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 847/1688] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:07:25.080864 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d279c-e32d-439d-b031-9075a87ab054 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:25.088372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2c7d279c-e32d-439d-b031-9075a87ab054 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:25.254218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:25.320848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d279c-e32d-439d-b031-9075a87ab054 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:25.338698 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cacc5-fc03-4100-b6d5-8a563999121e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:25.341308 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cacc5-fc03-4100-b6d5-8a563999121e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:25.341728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cacc5-fc03-4100-b6d5-8a563999121e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:25.342118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cacc5-fc03-4100-b6d5-8a563999121e admin admin] PATH_INFO: `/v3/projects/8362e5675331424982c4e018bbfea642` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:25.344686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cacc5-fc03-4100-b6d5-8a563999121e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8362e5675331424982c4e018bbfea642)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:25.350453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cacc5-fc03-4100-b6d5-8a563999121e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:25.513999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af0cacc5-fc03-4100-b6d5-8a563999121e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8362e5675331424982c4e018bbfea642', 'request_id': 'req-af0cacc5-fc03-4100-b6d5-8a563999121e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:25.560148 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af0cacc5-fc03-4100-b6d5-8a563999121e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8362e5675331424982c4e018bbfea642', 'request_id': 'req-af0cacc5-fc03-4100-b6d5-8a563999121e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:25.581269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af0cacc5-fc03-4100-b6d5-8a563999121e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8362e5675331424982c4e018bbfea642', 'request_id': 'req-af0cacc5-fc03-4100-b6d5-8a563999121e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:25.609722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af0cacc5-fc03-4100-b6d5-8a563999121e admin admin] The token cache is being invalidate because project 8362e5675331424982c4e018bbfea642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:25.610134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af0cacc5-fc03-4100-b6d5-8a563999121e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:25.611810 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af0cacc5-fc03-4100-b6d5-8a563999121e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:25.616359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 842/1689] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:25 2022] DELETE /identity/v3/projects/8362e5675331424982c4e018bbfea642 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:25.621753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730768c1-9e1c-45b3-ae51-1448a7e2849b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:25.622288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730768c1-9e1c-45b3-ae51-1448a7e2849b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:25.622915 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730768c1-9e1c-45b3-ae51-1448a7e2849b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:25.713750 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-730768c1-9e1c-45b3-ae51-1448a7e2849b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:25.716044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-730768c1-9e1c-45b3-ae51-1448a7e2849b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:26.006321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 848/1690] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:26.014167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe8579-8e82-44f7-8170-bc3212ed89b6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:26.014884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe8579-8e82-44f7-8170-bc3212ed89b6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:26.014884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe8579-8e82-44f7-8170-bc3212ed89b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:26.114309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-00fe8579-8e82-44f7-8170-bc3212ed89b6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:26.125854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-00fe8579-8e82-44f7-8170-bc3212ed89b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:26.160066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 843/1691] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:26.163731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730768c1-9e1c-45b3-ae51-1448a7e2849b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:26.185364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730768c1-9e1c-45b3-ae51-1448a7e2849b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:26.213735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7cbd5-f943-4f18-8b61-44f3e9295870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:26.215407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7cbd5-f943-4f18-8b61-44f3e9295870 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:26.215906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7cbd5-f943-4f18-8b61-44f3e9295870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:26.217267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7cbd5-f943-4f18-8b61-44f3e9295870 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:26.218851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7cbd5-f943-4f18-8b61-44f3e9295870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:26.223527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7cbd5-f943-4f18-8b61-44f3e9295870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:26.233676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7cbd5-f943-4f18-8b61-44f3e9295870 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:26.293573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 849/1692] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:26.294818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe8579-8e82-44f7-8170-bc3212ed89b6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:26.311265 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe8579-8e82-44f7-8170-bc3212ed89b6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:26.347057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b03c4-cd0c-423c-9db4-74cfaf77bfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:26.347935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b03c4-cd0c-423c-9db4-74cfaf77bfbe admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:26.351094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b03c4-cd0c-423c-9db4-74cfaf77bfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:26.352753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b03c4-cd0c-423c-9db4-74cfaf77bfbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:26.354329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b03c4-cd0c-423c-9db4-74cfaf77bfbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:26.359645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b03c4-cd0c-423c-9db4-74cfaf77bfbe admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:26.473152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 844/1693] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:26 2022] POST /identity/v3/projects => generated 363 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:26.475066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7cbd5-f943-4f18-8b61-44f3e9295870 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:26.487397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:26.497570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7cbd5-f943-4f18-8b61-44f3e9295870 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:26.576003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b03c4-cd0c-423c-9db4-74cfaf77bfbe admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:26.578195 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-379b03c4-cd0c-423c-9db4-74cfaf77bfbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:26.754261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b919e4-2508-4bf6-88d8-991e8d63d555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:26.755524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b919e4-2508-4bf6-88d8-991e8d63d555 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:26.757069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b919e4-2508-4bf6-88d8-991e8d63d555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:26.759029 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b919e4-2508-4bf6-88d8-991e8d63d555 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:26.760249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b919e4-2508-4bf6-88d8-991e8d63d555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:26.769034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b919e4-2508-4bf6-88d8-991e8d63d555 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:26.827802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 850/1694] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:26 2022] POST /identity/v3/users => generated 282 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:07:26.831303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b919e4-2508-4bf6-88d8-991e8d63d555 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:26.857767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b03c4-cd0c-423c-9db4-74cfaf77bfbe admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:26.869556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b919e4-2508-4bf6-88d8-991e8d63d555 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:26.874241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe38bd-8b45-4a7d-91c3-1a90a7eb29b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:26.875044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe38bd-8b45-4a7d-91c3-1a90a7eb29b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:26.875327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe38bd-8b45-4a7d-91c3-1a90a7eb29b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:26.877211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe38bd-8b45-4a7d-91c3-1a90a7eb29b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:26.880929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-dbbe38bd-8b45-4a7d-91c3-1a90a7eb29b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:26.881408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651bd94-cba9-4c07-93e3-1c5f8de6f48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:26.882308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651bd94-cba9-4c07-93e3-1c5f8de6f48b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:26.882792 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651bd94-cba9-4c07-93e3-1c5f8de6f48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:26.883194 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651bd94-cba9-4c07-93e3-1c5f8de6f48b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:26.884098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651bd94-cba9-4c07-93e3-1c5f8de6f48b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:26.891502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651bd94-cba9-4c07-93e3-1c5f8de6f48b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:26.920635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 851/1695] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:26.921988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651bd94-cba9-4c07-93e3-1c5f8de6f48b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:26.946730 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651bd94-cba9-4c07-93e3-1c5f8de6f48b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:26.972468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c5823-acd1-49e2-8c43-14c8d18dacc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:26.973551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c5823-acd1-49e2-8c43-14c8d18dacc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:26.974065 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c5823-acd1-49e2-8c43-14c8d18dacc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:26.974576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c5823-acd1-49e2-8c43-14c8d18dacc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:26.975778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c5823-acd1-49e2-8c43-14c8d18dacc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:26.978481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c5823-acd1-49e2-8c43-14c8d18dacc0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:27.038384 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 852/1696] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:27.043513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c5823-acd1-49e2-8c43-14c8d18dacc0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:27.071981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe38bd-8b45-4a7d-91c3-1a90a7eb29b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:27.075754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c5823-acd1-49e2-8c43-14c8d18dacc0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:27.076382 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe38bd-8b45-4a7d-91c3-1a90a7eb29b0 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:27.100190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 845/1697] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:27.110610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cef9f-9094-419d-ba27-7da2121b6c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:27.110610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cef9f-9094-419d-ba27-7da2121b6c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:27.110610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cef9f-9094-419d-ba27-7da2121b6c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:27.110610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cef9f-9094-419d-ba27-7da2121b6c75 admin admin] PATH_INFO: `/v3/projects/f74bd4caec0949b59ede43acfb0bcc1f/users/b6bf87cb99e04cdfab0d0c9552aecf04/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:27.149435 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cef9f-9094-419d-ba27-7da2121b6c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74bd4caec0949b59ede43acfb0bcc1f, user_id=b6bf87cb99e04cdfab0d0c9552aecf04, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:27.153888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cef9f-9094-419d-ba27-7da2121b6c75 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:27.188829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 853/1698] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:27 2022] PUT /identity/v3/projects/f74bd4caec0949b59ede43acfb0bcc1f/users/b6bf87cb99e04cdfab0d0c9552aecf04/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:27.200666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7d9f6-a2d0-4f31-a2a7-bcadbfd1b772 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:27.201135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7d9f6-a2d0-4f31-a2a7-bcadbfd1b772 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:27.201511 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7d9f6-a2d0-4f31-a2a7-bcadbfd1b772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:27.209461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:27.345286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4787b-ce9c-4aff-ba2e-8bd17bf621c0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:27.345695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4787b-ce9c-4aff-ba2e-8bd17bf621c0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:27.346091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4787b-ce9c-4aff-ba2e-8bd17bf621c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:27.391314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b9d7d9f6-a2d0-4f31-a2a7-bcadbfd1b772 None None] MFA Rules not processed for user `b6bf87cb99e04cdfab0d0c9552aecf04`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:27.399088 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b9d7d9f6-a2d0-4f31-a2a7-bcadbfd1b772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:27.428244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-47b4787b-ce9c-4aff-ba2e-8bd17bf621c0 None None] MFA Rules not processed for user `f330cdfd7d2b47a985d8dc76c19da9a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:27.434960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-47b4787b-ce9c-4aff-ba2e-8bd17bf621c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:27.736310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 846/1699] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:27 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 538 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:27.737346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7d9f6-a2d0-4f31-a2a7-bcadbfd1b772 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:27.741948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b9d7d9f6-a2d0-4f31-a2a7-bcadbfd1b772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:27.766116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 854/1700] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:27.773923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4787b-ce9c-4aff-ba2e-8bd17bf621c0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:27.955959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:28.016799 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7d9f6-a2d0-4f31-a2a7-bcadbfd1b772 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:28.030851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96fbd6-5e58-4e6b-8d67-053d8a04e5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:28.033347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96fbd6-5e58-4e6b-8d67-053d8a04e5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:28.033713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96fbd6-5e58-4e6b-8d67-053d8a04e5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:28.034033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96fbd6-5e58-4e6b-8d67-053d8a04e5e9 admin admin] PATH_INFO: `/v3/users/0b4caa892c3c4a3fb9ee06e6870f7160` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:28.035927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96fbd6-5e58-4e6b-8d67-053d8a04e5e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b4caa892c3c4a3fb9ee06e6870f7160)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:28.038068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96fbd6-5e58-4e6b-8d67-053d8a04e5e9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:28.082769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4787b-ce9c-4aff-ba2e-8bd17bf621c0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:28.120319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95753d-8ee2-4d0e-b97b-ee5b2720bdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:28.121610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95753d-8ee2-4d0e-b97b-ee5b2720bdc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:28.121902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95753d-8ee2-4d0e-b97b-ee5b2720bdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:28.122189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95753d-8ee2-4d0e-b97b-ee5b2720bdc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:28.123095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95753d-8ee2-4d0e-b97b-ee5b2720bdc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:28.125154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95753d-8ee2-4d0e-b97b-ee5b2720bdc7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:28.131307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af96fbd6-5e58-4e6b-8d67-053d8a04e5e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b4caa892c3c4a3fb9ee06e6870f7160'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:28.132948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af96fbd6-5e58-4e6b-8d67-053d8a04e5e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b4caa892c3c4a3fb9ee06e6870f7160'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:28.145291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af96fbd6-5e58-4e6b-8d67-053d8a04e5e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b4caa892c3c4a3fb9ee06e6870f7160'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:28.161032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af96fbd6-5e58-4e6b-8d67-053d8a04e5e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b4caa892c3c4a3fb9ee06e6870f7160'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:28.162431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af96fbd6-5e58-4e6b-8d67-053d8a04e5e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b4caa892c3c4a3fb9ee06e6870f7160'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:28.206581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 847/1701] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:27 2022] DELETE /identity/v3/users/0b4caa892c3c4a3fb9ee06e6870f7160 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:28.210017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2307c-92a2-4dfb-8d97-b1c1c0b7be11 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:28.210638 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2307c-92a2-4dfb-8d97-b1c1c0b7be11 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:28.211375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2307c-92a2-4dfb-8d97-b1c1c0b7be11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:28.229822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 855/1702] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:27 2022] POST /identity/v3/projects => generated 365 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:28.240099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad18081-bbf2-4000-bb9e-af45326ca0e5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:28.245751 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad18081-bbf2-4000-bb9e-af45326ca0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:28.246272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad18081-bbf2-4000-bb9e-af45326ca0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:28.288211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-99a2307c-92a2-4dfb-8d97-b1c1c0b7be11 None None] MFA Rules not processed for user `f330cdfd7d2b47a985d8dc76c19da9a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:28.293580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-99a2307c-92a2-4dfb-8d97-b1c1c0b7be11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:28.314893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-1ad18081-bbf2-4000-bb9e-af45326ca0e5 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:28.316193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1ad18081-bbf2-4000-bb9e-af45326ca0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:28.321224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:28.501357 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 856/1703] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:28.502568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad18081-bbf2-4000-bb9e-af45326ca0e5 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:28.504706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1ad18081-bbf2-4000-bb9e-af45326ca0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:28.518954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 848/1704] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:28.533006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2307c-92a2-4dfb-8d97-b1c1c0b7be11 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:28.542814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-99a2307c-92a2-4dfb-8d97-b1c1c0b7be11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:28.551348 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad18081-bbf2-4000-bb9e-af45326ca0e5 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:28.573604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4887602-c81d-4066-a236-1fd00516d6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:28.575183 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4887602-c81d-4066-a236-1fd00516d6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:28.575678 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4887602-c81d-4066-a236-1fd00516d6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:28.576161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4887602-c81d-4066-a236-1fd00516d6b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:28.577067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4887602-c81d-4066-a236-1fd00516d6b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:28.579317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4887602-c81d-4066-a236-1fd00516d6b2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:28.620212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 857/1705] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:28 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:28.621546 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4887602-c81d-4066-a236-1fd00516d6b2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:28.630378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d4887602-c81d-4066-a236-1fd00516d6b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:28.808956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2307c-92a2-4dfb-8d97-b1c1c0b7be11 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:28.820728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4887602-c81d-4066-a236-1fd00516d6b2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:28.829498 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f290c9b-cc8e-401a-b850-48e3ce982e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:28.831015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f290c9b-cc8e-401a-b850-48e3ce982e26 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:28.831604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f290c9b-cc8e-401a-b850-48e3ce982e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:28.832076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f290c9b-cc8e-401a-b850-48e3ce982e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:28.834855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb16bb-d654-4a73-9bd7-4c41c23b4e92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:28.835755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb16bb-d654-4a73-9bd7-4c41c23b4e92 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:28.835894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb16bb-d654-4a73-9bd7-4c41c23b4e92 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:28.836096 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb16bb-d654-4a73-9bd7-4c41c23b4e92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:28.842741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6beb16bb-d654-4a73-9bd7-4c41c23b4e92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:28.850725 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f290c9b-cc8e-401a-b850-48e3ce982e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:28.853243 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f290c9b-cc8e-401a-b850-48e3ce982e26 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:28.870071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb16bb-d654-4a73-9bd7-4c41c23b4e92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:28.872416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb16bb-d654-4a73-9bd7-4c41c23b4e92 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:28.875182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 858/1706] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:28.876573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f290c9b-cc8e-401a-b850-48e3ce982e26 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:28.890666 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f290c9b-cc8e-401a-b850-48e3ce982e26 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:28.899909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707ff3e-3f39-4d76-9f37-ff8c27ffda45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:28.900741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707ff3e-3f39-4d76-9f37-ff8c27ffda45 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:28.901023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707ff3e-3f39-4d76-9f37-ff8c27ffda45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:28.901300 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707ff3e-3f39-4d76-9f37-ff8c27ffda45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:28.902374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707ff3e-3f39-4d76-9f37-ff8c27ffda45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:28.906146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707ff3e-3f39-4d76-9f37-ff8c27ffda45 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:28.907811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 849/1707] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:07:28.926038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 859/1708] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:28.930644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb16bb-d654-4a73-9bd7-4c41c23b4e92 service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:28.946600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb16bb-d654-4a73-9bd7-4c41c23b4e92 service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:28.960203 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee59b17-42d7-4f67-80e9-34c594f8f8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:28.961127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee59b17-42d7-4f67-80e9-34c594f8f8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:28.961410 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee59b17-42d7-4f67-80e9-34c594f8f8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:28.961709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee59b17-42d7-4f67-80e9-34c594f8f8c9 admin admin] PATH_INFO: `/v3/projects/e3462f63802848e49d2be9d05b39cc70/users/30e75169ab934d3495260d44618d134f/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:28.991547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee59b17-42d7-4f67-80e9-34c594f8f8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3462f63802848e49d2be9d05b39cc70, user_id=30e75169ab934d3495260d44618d134f, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:28.994813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee59b17-42d7-4f67-80e9-34c594f8f8c9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:29.030270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 850/1709] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:28 2022] PUT /identity/v3/projects/e3462f63802848e49d2be9d05b39cc70/users/30e75169ab934d3495260d44618d134f/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:29.031435 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707ff3e-3f39-4d76-9f37-ff8c27ffda45 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:29.069627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707ff3e-3f39-4d76-9f37-ff8c27ffda45 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:29.191086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:29.269541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148236f3-d624-46a3-b361-df66bbb75942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:29.270307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148236f3-d624-46a3-b361-df66bbb75942 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:29.270604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148236f3-d624-46a3-b361-df66bbb75942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:29.270913 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148236f3-d624-46a3-b361-df66bbb75942 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:29.271739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148236f3-d624-46a3-b361-df66bbb75942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:29.276422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148236f3-d624-46a3-b361-df66bbb75942 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:29.310282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 860/1710] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:07:29.314570 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee59b17-42d7-4f67-80e9-34c594f8f8c9 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:29.333286 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee59b17-42d7-4f67-80e9-34c594f8f8c9 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:29.347622 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd56ca0-a048-4bd0-88ce-a8bcaa323c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:29.351217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd56ca0-a048-4bd0-88ce-a8bcaa323c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:29.351777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd56ca0-a048-4bd0-88ce-a8bcaa323c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:29.352699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd56ca0-a048-4bd0-88ce-a8bcaa323c19 admin admin] PATH_INFO: `/v3/projects/e3462f63802848e49d2be9d05b39cc70/users/30e75169ab934d3495260d44618d134f/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:29.360280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd56ca0-a048-4bd0-88ce-a8bcaa323c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3462f63802848e49d2be9d05b39cc70, user_id=30e75169ab934d3495260d44618d134f, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:29.368317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd56ca0-a048-4bd0-88ce-a8bcaa323c19 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:29.385879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986337d-2949-4c8b-b882-0ac6c31057c7 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:29.386178 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986337d-2949-4c8b-b882-0ac6c31057c7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:29.387045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986337d-2949-4c8b-b882-0ac6c31057c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:29.408313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 851/1711] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:29 2022] PUT /identity/v3/projects/e3462f63802848e49d2be9d05b39cc70/users/30e75169ab934d3495260d44618d134f/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:29.412268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499213c-bdc6-478c-a898-3da77397dcbc None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:29.413793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499213c-bdc6-478c-a898-3da77397dcbc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:29.414981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499213c-bdc6-478c-a898-3da77397dcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:29.472292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0986337d-2949-4c8b-b882-0ac6c31057c7 None None] MFA Rules not processed for user `bf3817e18de149aba43a2f65f37d63f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:29.473443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0986337d-2949-4c8b-b882-0ac6c31057c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:29.480885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5499213c-bdc6-478c-a898-3da77397dcbc None None] MFA Rules not processed for user `30e75169ab934d3495260d44618d134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:29.484848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5499213c-bdc6-478c-a898-3da77397dcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:29.619436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 861/1712] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:29.630218 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986337d-2949-4c8b-b882-0ac6c31057c7 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:29.635877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0986337d-2949-4c8b-b882-0ac6c31057c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:29.639844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:29.717381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 852/1713] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:29.729891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499213c-bdc6-478c-a898-3da77397dcbc None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:29.731593 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5499213c-bdc6-478c-a898-3da77397dcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:29.854816 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986337d-2949-4c8b-b882-0ac6c31057c7 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:29.872249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499213c-bdc6-478c-a898-3da77397dcbc None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:29.878016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a16c06-2fc0-4a6e-881f-2747db78d974 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:29.878831 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a16c06-2fc0-4a6e-881f-2747db78d974 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:29.879127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a16c06-2fc0-4a6e-881f-2747db78d974 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:29.879445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a16c06-2fc0-4a6e-881f-2747db78d974 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:29.882288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79a63b-b2af-4246-a281-51e42d511ce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:29.882983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79a63b-b2af-4246-a281-51e42d511ce8 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:29.883274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79a63b-b2af-4246-a281-51e42d511ce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:29.883586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79a63b-b2af-4246-a281-51e42d511ce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:29.901835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a16c06-2fc0-4a6e-881f-2747db78d974 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:29.906474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a16c06-2fc0-4a6e-881f-2747db78d974 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:29.915973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79a63b-b2af-4246-a281-51e42d511ce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:29.920377 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79a63b-b2af-4246-a281-51e42d511ce8 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:29.937489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 853/1714] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:29 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:29.940232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79a63b-b2af-4246-a281-51e42d511ce8 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:29.944209 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 862/1715] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:29.948754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0a79a63b-b2af-4246-a281-51e42d511ce8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:30.110906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a16c06-2fc0-4a6e-881f-2747db78d974 service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:30.130289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a16c06-2fc0-4a6e-881f-2747db78d974 service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:30.134559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442114e-d9eb-437d-902a-b5919f6c4c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:30.135496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442114e-d9eb-437d-902a-b5919f6c4c05 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:30.135953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442114e-d9eb-437d-902a-b5919f6c4c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:30.136409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442114e-d9eb-437d-902a-b5919f6c4c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:30.143556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3442114e-d9eb-437d-902a-b5919f6c4c05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:30.161674 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79a63b-b2af-4246-a281-51e42d511ce8 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:30.184337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:30.185134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:30.185423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:30.185932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67 admin admin] PATH_INFO: `/v3/projects/88118eaad2474baa9a9039d13a705efe` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:30.195525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88118eaad2474baa9a9039d13a705efe)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:30.198195 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:30.274578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:30.304589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442114e-d9eb-437d-902a-b5919f6c4c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:30.306337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442114e-d9eb-437d-902a-b5919f6c4c05 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:30.332816 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 863/1716] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:30 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:07:30.333857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88118eaad2474baa9a9039d13a705efe', 'request_id': 'req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:30.335722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47357de0-1549-4b7a-8bc2-5c61f30a9716 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:30.337071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47357de0-1549-4b7a-8bc2-5c61f30a9716 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:30.337757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47357de0-1549-4b7a-8bc2-5c61f30a9716 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:30.342748 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 864/1717] 10.222.0.75 () {54 vars in 912 bytes} [Thu Jul 14 11:07:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:07:30.345913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88118eaad2474baa9a9039d13a705efe', 'request_id': 'req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:30.349174 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ab857-c775-4728-9e6c-dfcf4f220daf None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:30.349629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ab857-c775-4728-9e6c-dfcf4f220daf None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:30.350084 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ab857-c775-4728-9e6c-dfcf4f220daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:30.369457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88118eaad2474baa9a9039d13a705efe', 'request_id': 'req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:30.398542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67 admin admin] The token cache is being invalidate because project 88118eaad2474baa9a9039d13a705efe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:30.399574 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:30.403065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-46d9f693-8ba3-41b7-a39f-4c0e0af1da67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:30.408341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 854/1718] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:29 2022] DELETE /identity/v3/projects/88118eaad2474baa9a9039d13a705efe => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:30.412902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6b7a4-d217-4d6e-83a7-cae7c9e262ab None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:30.413443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6b7a4-d217-4d6e-83a7-cae7c9e262ab None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:30.413869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6b7a4-d217-4d6e-83a7-cae7c9e262ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:30.449882 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e49ab857-c775-4728-9e6c-dfcf4f220daf None None] MFA Rules not processed for user `dadd485b7cac42f4aa079cab91973c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:30.453883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e49ab857-c775-4728-9e6c-dfcf4f220daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:30.456545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-02e6b7a4-d217-4d6e-83a7-cae7c9e262ab None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:30.460654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-02e6b7a4-d217-4d6e-83a7-cae7c9e262ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:30.507472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:30.598058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 865/1719] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:07:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:30.612340 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 855/1720] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:30.620033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d40337-88f6-4bbd-855e-926a67c205c6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:30.620323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d40337-88f6-4bbd-855e-926a67c205c6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:30.620603 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d40337-88f6-4bbd-855e-926a67c205c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:30.672443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d3d40337-88f6-4bbd-855e-926a67c205c6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:30.674431 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d3d40337-88f6-4bbd-855e-926a67c205c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:30.698705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 866/1721] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:30.702970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6b7a4-d217-4d6e-83a7-cae7c9e262ab None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:30.709835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d40337-88f6-4bbd-855e-926a67c205c6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:30.714954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d3d40337-88f6-4bbd-855e-926a67c205c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:30.732984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6b7a4-d217-4d6e-83a7-cae7c9e262ab None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:30.754887 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d00a0c-6b1f-481a-ba79-7cae798afe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:30.755829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d00a0c-6b1f-481a-ba79-7cae798afe7b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:30.756012 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d00a0c-6b1f-481a-ba79-7cae798afe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:30.756382 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d00a0c-6b1f-481a-ba79-7cae798afe7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:30.757168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d00a0c-6b1f-481a-ba79-7cae798afe7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:30.757495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d00a0c-6b1f-481a-ba79-7cae798afe7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:30.759249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d00a0c-6b1f-481a-ba79-7cae798afe7b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:30.781667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d40337-88f6-4bbd-855e-926a67c205c6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:30.799894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28e9fb-1ae7-4b57-8648-1be85946eb6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:30.800671 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28e9fb-1ae7-4b57-8648-1be85946eb6b service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:30.800957 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28e9fb-1ae7-4b57-8648-1be85946eb6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:30.801239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28e9fb-1ae7-4b57-8648-1be85946eb6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:30.820431 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28e9fb-1ae7-4b57-8648-1be85946eb6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:30.822167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28e9fb-1ae7-4b57-8648-1be85946eb6b service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:30.829121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 856/1722] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:30.831187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d00a0c-6b1f-481a-ba79-7cae798afe7b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:30.851656 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 867/1723] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:30.854213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d00a0c-6b1f-481a-ba79-7cae798afe7b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:30.860232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b554e-ef6f-4414-8843-252839434fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:30.861029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b554e-ef6f-4414-8843-252839434fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:30.861372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b554e-ef6f-4414-8843-252839434fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:30.861654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b554e-ef6f-4414-8843-252839434fe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:30.862336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b554e-ef6f-4414-8843-252839434fe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:30.864529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b554e-ef6f-4414-8843-252839434fe3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:30.916077 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 857/1724] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:30 2022] POST /identity/v3/projects => generated 363 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:30.917897 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28e9fb-1ae7-4b57-8648-1be85946eb6b service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:30.939906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28e9fb-1ae7-4b57-8648-1be85946eb6b service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:31.042368 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9156b-0f94-472b-8349-4b580caffbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:31.046267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9156b-0f94-472b-8349-4b580caffbdc admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:31.046558 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9156b-0f94-472b-8349-4b580caffbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:31.047937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9156b-0f94-472b-8349-4b580caffbdc admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:31.050679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9156b-0f94-472b-8349-4b580caffbdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:31.056627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9156b-0f94-472b-8349-4b580caffbdc admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:31.088143 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 868/1725] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:30 2022] POST /identity/v3/users => generated 282 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:31.093058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b554e-ef6f-4414-8843-252839434fe3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:31.106636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b554e-ef6f-4414-8843-252839434fe3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:31.114216 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a16430-8eda-4356-8e84-de6d3a0ffcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:31.115476 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a16430-8eda-4356-8e84-de6d3a0ffcc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:31.115765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a16430-8eda-4356-8e84-de6d3a0ffcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:31.116045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a16430-8eda-4356-8e84-de6d3a0ffcc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:31.116813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a16430-8eda-4356-8e84-de6d3a0ffcc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:31.118900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a16430-8eda-4356-8e84-de6d3a0ffcc3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:31.139571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 858/1726] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:31.140812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9156b-0f94-472b-8349-4b580caffbdc admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:31.151657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9156b-0f94-472b-8349-4b580caffbdc admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:31.156654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f02e22-1932-4769-97d7-823f07311484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:31.157342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f02e22-1932-4769-97d7-823f07311484 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:31.157955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f02e22-1932-4769-97d7-823f07311484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:31.158245 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f02e22-1932-4769-97d7-823f07311484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:31.158989 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f02e22-1932-4769-97d7-823f07311484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:31.160843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f02e22-1932-4769-97d7-823f07311484 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:31.173289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 869/1727] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:31.177485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a16430-8eda-4356-8e84-de6d3a0ffcc3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:31.190413 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a16430-8eda-4356-8e84-de6d3a0ffcc3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:31.200257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccda0a9-e754-4d31-9792-4a8099a208cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:31.201176 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccda0a9-e754-4d31-9792-4a8099a208cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:31.201529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccda0a9-e754-4d31-9792-4a8099a208cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:31.201747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccda0a9-e754-4d31-9792-4a8099a208cf admin admin] PATH_INFO: `/v3/projects/2d78139f10f041bbab2fc9e013895747/users/1427c92d53a748668b3bcb2512d9ccad/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:31.224359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccda0a9-e754-4d31-9792-4a8099a208cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d78139f10f041bbab2fc9e013895747, user_id=1427c92d53a748668b3bcb2512d9ccad, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:31.227727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccda0a9-e754-4d31-9792-4a8099a208cf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:31.247617 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 859/1728] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:31 2022] PUT /identity/v3/projects/2d78139f10f041bbab2fc9e013895747/users/1427c92d53a748668b3bcb2512d9ccad/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:31.258652 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a99ea-42c0-4a21-8b05-fd69b1e11888 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:31.258967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a99ea-42c0-4a21-8b05-fd69b1e11888 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:31.259227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a99ea-42c0-4a21-8b05-fd69b1e11888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:31.309149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-dd8a99ea-42c0-4a21-8b05-fd69b1e11888 None None] MFA Rules not processed for user `1427c92d53a748668b3bcb2512d9ccad`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:31.310605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-dd8a99ea-42c0-4a21-8b05-fd69b1e11888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:31.361857 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:31.414305 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 870/1729] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:31.419132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccda0a9-e754-4d31-9792-4a8099a208cf admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:31.509722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccda0a9-e754-4d31-9792-4a8099a208cf admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:31.516196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0c06d-d7d6-48b9-b0ac-35d4fe27edc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:31.516667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0c06d-d7d6-48b9-b0ac-35d4fe27edc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:31.516961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0c06d-d7d6-48b9-b0ac-35d4fe27edc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:31.517243 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0c06d-d7d6-48b9-b0ac-35d4fe27edc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:31.518009 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0c06d-d7d6-48b9-b0ac-35d4fe27edc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:31.520031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0c06d-d7d6-48b9-b0ac-35d4fe27edc8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:31.557014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 860/1730] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:31 2022] POST /identity/v3/projects => generated 361 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:07:31.559731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a99ea-42c0-4a21-8b05-fd69b1e11888 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:31.575481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a99ea-42c0-4a21-8b05-fd69b1e11888 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:31.659437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438b7e0-b880-4d7f-8ec2-356f90776061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:31.660066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438b7e0-b880-4d7f-8ec2-356f90776061 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:31.660365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438b7e0-b880-4d7f-8ec2-356f90776061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:31.661002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438b7e0-b880-4d7f-8ec2-356f90776061 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:31.661419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438b7e0-b880-4d7f-8ec2-356f90776061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:31.663533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438b7e0-b880-4d7f-8ec2-356f90776061 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:31.690072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 871/1731] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:31 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:31.695733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0c06d-d7d6-48b9-b0ac-35d4fe27edc8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:31.710647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0c06d-d7d6-48b9-b0ac-35d4fe27edc8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:31.718102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd520c42-bb64-43b7-ad75-a35a3eeeb210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:31.719069 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd520c42-bb64-43b7-ad75-a35a3eeeb210 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:31.719657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd520c42-bb64-43b7-ad75-a35a3eeeb210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:31.719765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd520c42-bb64-43b7-ad75-a35a3eeeb210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:31.721262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd520c42-bb64-43b7-ad75-a35a3eeeb210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:31.724251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd520c42-bb64-43b7-ad75-a35a3eeeb210 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:31.752035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 861/1732] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:31.754726 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438b7e0-b880-4d7f-8ec2-356f90776061 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:31.765547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438b7e0-b880-4d7f-8ec2-356f90776061 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:31.772565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cce089-fefd-4b87-968d-4d8f935af1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:31.773274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cce089-fefd-4b87-968d-4d8f935af1d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:31.773968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cce089-fefd-4b87-968d-4d8f935af1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:31.773968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cce089-fefd-4b87-968d-4d8f935af1d1 admin admin] PATH_INFO: `/v3/projects/8e42462377304596979258660371bdbf/users/af705f571d09432b9044c9f5f9e9ca27/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:31.790555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cce089-fefd-4b87-968d-4d8f935af1d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e42462377304596979258660371bdbf, user_id=af705f571d09432b9044c9f5f9e9ca27, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:31.793937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cce089-fefd-4b87-968d-4d8f935af1d1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:31.848575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 872/1733] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:31 2022] PUT /identity/v3/projects/8e42462377304596979258660371bdbf/users/af705f571d09432b9044c9f5f9e9ca27/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:31.864316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bc80c-3444-4bca-9915-ce0efa9a52fd None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:31.864924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bc80c-3444-4bca-9915-ce0efa9a52fd None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:31.865426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bc80c-3444-4bca-9915-ce0efa9a52fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:31.867434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd520c42-bb64-43b7-ad75-a35a3eeeb210 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:31.911810 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd520c42-bb64-43b7-ad75-a35a3eeeb210 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:31.938938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-710bc80c-3444-4bca-9915-ce0efa9a52fd None None] MFA Rules not processed for user `cb80e0c64c3348cfac5ad2c4f82fb606`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:31.941876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-710bc80c-3444-4bca-9915-ce0efa9a52fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:31.986784 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:32.073064 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca496eae-5b8c-46fb-8d27-f252a64c48a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:32.073813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca496eae-5b8c-46fb-8d27-f252a64c48a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:32.074104 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca496eae-5b8c-46fb-8d27-f252a64c48a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:32.074383 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca496eae-5b8c-46fb-8d27-f252a64c48a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:32.075145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca496eae-5b8c-46fb-8d27-f252a64c48a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:32.087430 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca496eae-5b8c-46fb-8d27-f252a64c48a4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:32.139569 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 862/1734] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:32.140617 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca496eae-5b8c-46fb-8d27-f252a64c48a4 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:32.146700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ca496eae-5b8c-46fb-8d27-f252a64c48a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:32.195077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 873/1735] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:32.195077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bc80c-3444-4bca-9915-ce0efa9a52fd None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:32.204848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bc80c-3444-4bca-9915-ce0efa9a52fd None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:32.205541 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca496eae-5b8c-46fb-8d27-f252a64c48a4 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:32.227445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae366dc-d500-4983-a6a4-de21804f7b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:32.227775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d999a18-279a-434b-ad9a-7810f1d735c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:32.229972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d999a18-279a-434b-ad9a-7810f1d735c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:32.230852 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d999a18-279a-434b-ad9a-7810f1d735c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:32.231143 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d999a18-279a-434b-ad9a-7810f1d735c7 admin admin] PATH_INFO: `/v3/projects/8e42462377304596979258660371bdbf/users/af705f571d09432b9044c9f5f9e9ca27/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:32.238181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae366dc-d500-4983-a6a4-de21804f7b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:32.238991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae366dc-d500-4983-a6a4-de21804f7b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:32.239287 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae366dc-d500-4983-a6a4-de21804f7b9a admin admin] PATH_INFO: `/v3/users/02052e514aaa43d9af166f2bf092d93c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:32.242710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae366dc-d500-4983-a6a4-de21804f7b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=02052e514aaa43d9af166f2bf092d93c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:32.243502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d999a18-279a-434b-ad9a-7810f1d735c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e42462377304596979258660371bdbf, user_id=af705f571d09432b9044c9f5f9e9ca27, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:32.244794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae366dc-d500-4983-a6a4-de21804f7b9a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:32.262306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d999a18-279a-434b-ad9a-7810f1d735c7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:32.315426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 874/1736] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:32 2022] PUT /identity/v3/projects/8e42462377304596979258660371bdbf/users/af705f571d09432b9044c9f5f9e9ca27/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:32.315426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7daa1-1242-4563-8113-7215df373a22 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:32.315426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7daa1-1242-4563-8113-7215df373a22 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:32.315426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7daa1-1242-4563-8113-7215df373a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:32.373664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a1e7daa1-1242-4563-8113-7215df373a22 None None] MFA Rules not processed for user `cb80e0c64c3348cfac5ad2c4f82fb606`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:32.379828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a1e7daa1-1242-4563-8113-7215df373a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:32.383899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3ae366dc-d500-4983-a6a4-de21804f7b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02052e514aaa43d9af166f2bf092d93c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:32.385561 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3ae366dc-d500-4983-a6a4-de21804f7b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02052e514aaa43d9af166f2bf092d93c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:32.400427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3ae366dc-d500-4983-a6a4-de21804f7b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02052e514aaa43d9af166f2bf092d93c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:32.406908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:32.421394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3ae366dc-d500-4983-a6a4-de21804f7b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02052e514aaa43d9af166f2bf092d93c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:32.422885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3ae366dc-d500-4983-a6a4-de21804f7b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02052e514aaa43d9af166f2bf092d93c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:32.477371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 863/1737] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:32 2022] DELETE /identity/v3/users/02052e514aaa43d9af166f2bf092d93c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:32.489902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7bff7-a48e-427f-9854-c3b0509d953c None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:32.490340 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7bff7-a48e-427f-9854-c3b0509d953c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:32.491857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7bff7-a48e-427f-9854-c3b0509d953c None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:32.496266 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 864/1738] 10.222.0.75 () {54 vars in 960 bytes} [Thu Jul 14 11:07:32 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:07:32.497565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ae16e-ec73-4ded-b6da-c93734545ce6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:32.497858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ae16e-ec73-4ded-b6da-c93734545ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:32.498206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ae16e-ec73-4ded-b6da-c93734545ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:32.596572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 875/1739] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:32.598412 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6935a70-46c9-48f9-8cd7-643bad6db434 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:32.598675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6935a70-46c9-48f9-8cd7-643bad6db434 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:32.599690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6935a70-46c9-48f9-8cd7-643bad6db434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:32.611434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3b6ae16e-ec73-4ded-b6da-c93734545ce6 None None] MFA Rules not processed for user `af705f571d09432b9044c9f5f9e9ca27`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:32.612741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3b6ae16e-ec73-4ded-b6da-c93734545ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:32.645361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f6935a70-46c9-48f9-8cd7-643bad6db434 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:32.646853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f6935a70-46c9-48f9-8cd7-643bad6db434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:32.841297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 876/1740] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:32.853585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24d6ae-d9c9-4a69-bea2-ad83b6e5bc9d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:32.857550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24d6ae-d9c9-4a69-bea2-ad83b6e5bc9d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:32.859957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 865/1741] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:32 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:32.862556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24d6ae-d9c9-4a69-bea2-ad83b6e5bc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:32.865719 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ae16e-ec73-4ded-b6da-c93734545ce6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:32.884261 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3b6ae16e-ec73-4ded-b6da-c93734545ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:32.941316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:32.975310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6e24d6ae-d9c9-4a69-bea2-ad83b6e5bc9d None None] MFA Rules not processed for user `0fc1f2abce4845219741f4f48bf32d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:32.983278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6e24d6ae-d9c9-4a69-bea2-ad83b6e5bc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:33.087844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ae16e-ec73-4ded-b6da-c93734545ce6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:33.102081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d898eaf-1e84-4b7b-90ea-782c7f68e23a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:33.102954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d898eaf-1e84-4b7b-90ea-782c7f68e23a service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:33.103246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d898eaf-1e84-4b7b-90ea-782c7f68e23a service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:33.103555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d898eaf-1e84-4b7b-90ea-782c7f68e23a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:33.108459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1d898eaf-1e84-4b7b-90ea-782c7f68e23a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:33.231542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 877/1742] 10.222.0.75 () {58 vars in 1057 bytes} [Thu Jul 14 11:07:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:33.232811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24d6ae-d9c9-4a69-bea2-ad83b6e5bc9d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:33.234943 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6e24d6ae-d9c9-4a69-bea2-ad83b6e5bc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:33.291666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d898eaf-1e84-4b7b-90ea-782c7f68e23a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:33.295816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d898eaf-1e84-4b7b-90ea-782c7f68e23a service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:33.333524 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 866/1743] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:33.334397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d898eaf-1e84-4b7b-90ea-782c7f68e23a service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:33.343475 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1d898eaf-1e84-4b7b-90ea-782c7f68e23a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:33.422060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24d6ae-d9c9-4a69-bea2-ad83b6e5bc9d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:33.428823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64825e61-ef3e-4d12-8bbd-a03672b6a174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:33.429354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64825e61-ef3e-4d12-8bbd-a03672b6a174 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:33.429909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64825e61-ef3e-4d12-8bbd-a03672b6a174 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:33.430189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64825e61-ef3e-4d12-8bbd-a03672b6a174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:33.444022 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64825e61-ef3e-4d12-8bbd-a03672b6a174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:33.445981 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64825e61-ef3e-4d12-8bbd-a03672b6a174 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:33.451977 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:33.463715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 878/1744] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:33.464654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64825e61-ef3e-4d12-8bbd-a03672b6a174 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:33.484929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64825e61-ef3e-4d12-8bbd-a03672b6a174 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:33.502505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ae4ad-46a2-4bea-8e51-34d12da4259a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:33.503186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ae4ad-46a2-4bea-8e51-34d12da4259a service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:33.503525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ae4ad-46a2-4bea-8e51-34d12da4259a service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:33.503809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ae4ad-46a2-4bea-8e51-34d12da4259a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:33.507856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b75ae4ad-46a2-4bea-8e51-34d12da4259a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:33.543694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ae4ad-46a2-4bea-8e51-34d12da4259a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:33.545857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d898eaf-1e84-4b7b-90ea-782c7f68e23a service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:33.548713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ae4ad-46a2-4bea-8e51-34d12da4259a service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:33.551322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08b47b-1d17-423f-9e40-641ffb1d54f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:33.552670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b47b-1d17-423f-9e40-641ffb1d54f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:33.553127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b47b-1d17-423f-9e40-641ffb1d54f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:33.553732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b47b-1d17-423f-9e40-641ffb1d54f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:33.565985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08b47b-1d17-423f-9e40-641ffb1d54f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:33.567796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08b47b-1d17-423f-9e40-641ffb1d54f6 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:33.595869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 867/1745] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:33 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:33.603602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a834e7-8148-4391-8da4-6a8a67a711c3 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:33.611887 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a834e7-8148-4391-8da4-6a8a67a711c3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:33.612509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a834e7-8148-4391-8da4-6a8a67a711c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:33.615568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 879/1746] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:33.668886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-46a834e7-8148-4391-8da4-6a8a67a711c3 None None] MFA Rules not processed for user `fffbf852ddc9455f8a68f80ad118e4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:33.671868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-46a834e7-8148-4391-8da4-6a8a67a711c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:33.796614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 868/1747] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:34.863991 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ae4ad-46a2-4bea-8e51-34d12da4259a service placement] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:34.867050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b75ae4ad-46a2-4bea-8e51-34d12da4259a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:34.917535 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ae4ad-46a2-4bea-8e51-34d12da4259a service placement] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:34.937576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:34.938654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:34.938993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:34.939225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] PATH_INFO: `/v3/users/1427c92d53a748668b3bcb2512d9ccad` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:34.941851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1427c92d53a748668b3bcb2512d9ccad)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:34.947937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:35.106842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1427c92d53a748668b3bcb2512d9ccad'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:35.128978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1427c92d53a748668b3bcb2512d9ccad'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:35.129626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1427c92d53a748668b3bcb2512d9ccad'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:35.147079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1427c92d53a748668b3bcb2512d9ccad'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:35.150950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1427c92d53a748668b3bcb2512d9ccad'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:35.227597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 880/1748] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:34 2022] DELETE /identity/v3/users/1427c92d53a748668b3bcb2512d9ccad => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:35.231961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686dbc3-234a-4fa8-bac1-ef773ee6bff5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:35.232826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686dbc3-234a-4fa8-bac1-ef773ee6bff5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:35.233249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686dbc3-234a-4fa8-bac1-ef773ee6bff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:35.240402 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:35.283557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f686dbc3-234a-4fa8-bac1-ef773ee6bff5 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:35.288902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f686dbc3-234a-4fa8-bac1-ef773ee6bff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:35.443897 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:35.445969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:35.685042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 869/1749] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:35.698215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686dbc3-234a-4fa8-bac1-ef773ee6bff5 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:35.711641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f686dbc3-234a-4fa8-bac1-ef773ee6bff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:35.805868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b3f43-cd76-4e3e-9c47-3d0e807592d5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:35.829231 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ee10f-e473-4ff7-947d-ed6823293d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:35.830129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ee10f-e473-4ff7-947d-ed6823293d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:35.830407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ee10f-e473-4ff7-947d-ed6823293d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:35.830683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ee10f-e473-4ff7-947d-ed6823293d0f admin admin] PATH_INFO: `/v3/projects/b3c9c45844504b80b76887e8450188f8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:35.843712 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ee10f-e473-4ff7-947d-ed6823293d0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c9c45844504b80b76887e8450188f8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:35.858491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ee10f-e473-4ff7-947d-ed6823293d0f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:36.013841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686dbc3-234a-4fa8-bac1-ef773ee6bff5 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:36.040864 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91592f-41d7-47a3-b971-0f5d23be0a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:36.043106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91592f-41d7-47a3-b971-0f5d23be0a1b service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:36.043755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91592f-41d7-47a3-b971-0f5d23be0a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:36.044528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91592f-41d7-47a3-b971-0f5d23be0a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:36.059498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91592f-41d7-47a3-b971-0f5d23be0a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:36.061571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91592f-41d7-47a3-b971-0f5d23be0a1b service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:36.081526 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 870/1750] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:36.084279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955681a4-e783-420d-90de-41f99f914b08 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:36.084882 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955681a4-e783-420d-90de-41f99f914b08 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:36.085328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955681a4-e783-420d-90de-41f99f914b08 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:36.094890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 871/1751] 10.222.0.75 () {54 vars in 912 bytes} [Thu Jul 14 11:07:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:07:36.097135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955681a4-e783-420d-90de-41f99f914b08 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:36.116587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-955681a4-e783-420d-90de-41f99f914b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:36.126181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4f2ee10f-e473-4ff7-947d-ed6823293d0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c9c45844504b80b76887e8450188f8', 'request_id': 'req-4f2ee10f-e473-4ff7-947d-ed6823293d0f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:36.165786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4f2ee10f-e473-4ff7-947d-ed6823293d0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c9c45844504b80b76887e8450188f8', 'request_id': 'req-4f2ee10f-e473-4ff7-947d-ed6823293d0f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:36.177170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:36.183094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4f2ee10f-e473-4ff7-947d-ed6823293d0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c9c45844504b80b76887e8450188f8', 'request_id': 'req-4f2ee10f-e473-4ff7-947d-ed6823293d0f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:36.253172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4f2ee10f-e473-4ff7-947d-ed6823293d0f admin admin] The token cache is being invalidate because project b3c9c45844504b80b76887e8450188f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:36.253801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4f2ee10f-e473-4ff7-947d-ed6823293d0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:36.256733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4f2ee10f-e473-4ff7-947d-ed6823293d0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:36.263315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 881/1752] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:35 2022] DELETE /identity/v3/projects/b3c9c45844504b80b76887e8450188f8 => generated 0 bytes in 820 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:07:36.276447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8040749-3d55-4f60-b286-8feda0a95990 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:36.277107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8040749-3d55-4f60-b286-8feda0a95990 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:36.277546 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8040749-3d55-4f60-b286-8feda0a95990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:36.285694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955681a4-e783-420d-90de-41f99f914b08 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:36.290638 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057273cf-8796-40a6-b2fc-a0bbd54a7280 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:36.291711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057273cf-8796-40a6-b2fc-a0bbd54a7280 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:36.292242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057273cf-8796-40a6-b2fc-a0bbd54a7280 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:36.293393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057273cf-8796-40a6-b2fc-a0bbd54a7280 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:36.295856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-057273cf-8796-40a6-b2fc-a0bbd54a7280 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:36.341649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c8040749-3d55-4f60-b286-8feda0a95990 None None] MFA Rules not processed for user `dadd485b7cac42f4aa079cab91973c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:36.344149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c8040749-3d55-4f60-b286-8feda0a95990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:36.457099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057273cf-8796-40a6-b2fc-a0bbd54a7280 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:36.472652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057273cf-8796-40a6-b2fc-a0bbd54a7280 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:36.535240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 872/1753] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:36.543773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475e1ad-d515-4c74-894b-4ecaa316f6ee None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:36.544523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475e1ad-d515-4c74-894b-4ecaa316f6ee None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:36.546512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475e1ad-d515-4c74-894b-4ecaa316f6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:36.602547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 882/1754] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:07:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:36.604654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:36.609973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e37b5a-899b-40be-a3e6-046889b884f0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:36.612566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e37b5a-899b-40be-a3e6-046889b884f0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:36.614263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e37b5a-899b-40be-a3e6-046889b884f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:36.685257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5475e1ad-d515-4c74-894b-4ecaa316f6ee None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:36.688151 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5475e1ad-d515-4c74-894b-4ecaa316f6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:36.707500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-12e37b5a-899b-40be-a3e6-046889b884f0 None None] MFA Rules not processed for user `cb80e0c64c3348cfac5ad2c4f82fb606`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:36.715627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-12e37b5a-899b-40be-a3e6-046889b884f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:36.901789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 873/1755] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:36.903683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475e1ad-d515-4c74-894b-4ecaa316f6ee None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:36.905685 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5475e1ad-d515-4c74-894b-4ecaa316f6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:36.942015 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475e1ad-d515-4c74-894b-4ecaa316f6ee None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:36.953029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727e485-cafc-4ac3-8ea1-ee405ca1d8b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:36.954010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727e485-cafc-4ac3-8ea1-ee405ca1d8b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:36.954311 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727e485-cafc-4ac3-8ea1-ee405ca1d8b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:36.959861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 883/1756] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:36.962599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727e485-cafc-4ac3-8ea1-ee405ca1d8b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:36.964302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d8e86-5d1b-4c92-af61-b81709f19d15 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:36.965432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d8e86-5d1b-4c92-af61-b81709f19d15 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:36.965732 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d8e86-5d1b-4c92-af61-b81709f19d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:36.980562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727e485-cafc-4ac3-8ea1-ee405ca1d8b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:36.986314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727e485-cafc-4ac3-8ea1-ee405ca1d8b5 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.026346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c94d8e86-5d1b-4c92-af61-b81709f19d15 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:37.043437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c94d8e86-5d1b-4c92-af61-b81709f19d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:37.050376 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 874/1757] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:37.069098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e7141-5e6e-4f52-9de1-3095ffe15074 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.069537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e7141-5e6e-4f52-9de1-3095ffe15074 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.069928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e7141-5e6e-4f52-9de1-3095ffe15074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.090952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 884/1758] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:37.094923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d8e86-5d1b-4c92-af61-b81709f19d15 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.116538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d8e86-5d1b-4c92-af61-b81709f19d15 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.119005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6b5e7141-5e6e-4f52-9de1-3095ffe15074 None None] MFA Rules not processed for user `bf3817e18de149aba43a2f65f37d63f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:37.122679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6b5e7141-5e6e-4f52-9de1-3095ffe15074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:37.151544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f50f5-8785-4e21-95aa-b8f36d194d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.154717 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f50f5-8785-4e21-95aa-b8f36d194d1d admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.155757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f50f5-8785-4e21-95aa-b8f36d194d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.156246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f50f5-8785-4e21-95aa-b8f36d194d1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.160054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f50f5-8785-4e21-95aa-b8f36d194d1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:37.162638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f50f5-8785-4e21-95aa-b8f36d194d1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.166083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f50f5-8785-4e21-95aa-b8f36d194d1d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.217196 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 885/1759] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:07:37.221280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f50f5-8785-4e21-95aa-b8f36d194d1d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.241134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f50f5-8785-4e21-95aa-b8f36d194d1d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.260430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59ead7-23b7-4d2d-ba3e-e2477326c3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.262009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59ead7-23b7-4d2d-ba3e-e2477326c3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.262499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59ead7-23b7-4d2d-ba3e-e2477326c3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.264262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59ead7-23b7-4d2d-ba3e-e2477326c3c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.265290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59ead7-23b7-4d2d-ba3e-e2477326c3c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.269017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59ead7-23b7-4d2d-ba3e-e2477326c3c2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.290149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 875/1760] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:37.355757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 886/1761] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:37 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:07:37.360496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e7141-5e6e-4f52-9de1-3095ffe15074 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.379355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e7141-5e6e-4f52-9de1-3095ffe15074 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.579365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59ead7-23b7-4d2d-ba3e-e2477326c3c2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.582957 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6f59ead7-23b7-4d2d-ba3e-e2477326c3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:37.640281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e013ed-b3f9-4466-a97b-d3d2f8994dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.642347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e013ed-b3f9-4466-a97b-d3d2f8994dac admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.645089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e013ed-b3f9-4466-a97b-d3d2f8994dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.646497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e013ed-b3f9-4466-a97b-d3d2f8994dac admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.652352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e013ed-b3f9-4466-a97b-d3d2f8994dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.689996 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e013ed-b3f9-4466-a97b-d3d2f8994dac admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.859948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 876/1762] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:37 2022] POST /identity/v3/users => generated 287 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:37.867186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e013ed-b3f9-4466-a97b-d3d2f8994dac admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.898136 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e013ed-b3f9-4466-a97b-d3d2f8994dac admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.907252 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377fb93-3b8b-42cc-87d0-6dbdcdc5a20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.908283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377fb93-3b8b-42cc-87d0-6dbdcdc5a20d admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.908283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377fb93-3b8b-42cc-87d0-6dbdcdc5a20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.908559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377fb93-3b8b-42cc-87d0-6dbdcdc5a20d admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.909229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377fb93-3b8b-42cc-87d0-6dbdcdc5a20d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.925575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377fb93-3b8b-42cc-87d0-6dbdcdc5a20d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.951299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59ead7-23b7-4d2d-ba3e-e2477326c3c2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.954201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 877/1763] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:37.961775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.963283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.965758 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.969717 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] PATH_INFO: `/v3/projects/2d78139f10f041bbab2fc9e013895747` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.971258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377fb93-3b8b-42cc-87d0-6dbdcdc5a20d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.971666 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d78139f10f041bbab2fc9e013895747)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.974233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.976823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:37.997721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377fb93-3b8b-42cc-87d0-6dbdcdc5a20d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:38.010824 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de0669-30e9-4c81-ad5a-d824683e4b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:38.011931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de0669-30e9-4c81-ad5a-d824683e4b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:38.012608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de0669-30e9-4c81-ad5a-d824683e4b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:38.013088 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de0669-30e9-4c81-ad5a-d824683e4b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:38.014101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de0669-30e9-4c81-ad5a-d824683e4b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:38.021572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de0669-30e9-4c81-ad5a-d824683e4b72 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:38.071831 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 878/1764] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:38.074168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de0669-30e9-4c81-ad5a-d824683e4b72 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:38.086779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:38.097369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de0669-30e9-4c81-ad5a-d824683e4b72 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:38.112830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c513c-0a35-42c6-9fe8-d28b1a9627a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:38.116122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c513c-0a35-42c6-9fe8-d28b1a9627a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:38.116825 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c513c-0a35-42c6-9fe8-d28b1a9627a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:38.117367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c513c-0a35-42c6-9fe8-d28b1a9627a9 admin admin] PATH_INFO: `/v3/projects/02024f97891d489c8db7dac94182fe5d/users/3f7fd9aa8fb147098c6a32c11017f79d/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:38.146066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c513c-0a35-42c6-9fe8-d28b1a9627a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02024f97891d489c8db7dac94182fe5d, user_id=3f7fd9aa8fb147098c6a32c11017f79d, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:38.150172 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c513c-0a35-42c6-9fe8-d28b1a9627a9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:38.175434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 879/1765] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:38 2022] PUT /identity/v3/projects/02024f97891d489c8db7dac94182fe5d/users/3f7fd9aa8fb147098c6a32c11017f79d/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:38.193417 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e26fe-601c-4ed0-9f4a-47b55594ed28 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:38.193756 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e26fe-601c-4ed0-9f4a-47b55594ed28 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:38.194267 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e26fe-601c-4ed0-9f4a-47b55594ed28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:38.243111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-fc0e26fe-601c-4ed0-9f4a-47b55594ed28 None None] MFA Rules not processed for user `3f7fd9aa8fb147098c6a32c11017f79d`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:38.244849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d78139f10f041bbab2fc9e013895747', 'request_id': 'req-88510ede-a02d-40c5-b63d-32690243bb09'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:38.245327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fc0e26fe-601c-4ed0-9f4a-47b55594ed28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:38.283848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d78139f10f041bbab2fc9e013895747', 'request_id': 'req-88510ede-a02d-40c5-b63d-32690243bb09'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:38.320176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d78139f10f041bbab2fc9e013895747', 'request_id': 'req-88510ede-a02d-40c5-b63d-32690243bb09'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:38.329186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 880/1766] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:38.333563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e26fe-601c-4ed0-9f4a-47b55594ed28 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:38.353288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] The token cache is being invalidate because project 2d78139f10f041bbab2fc9e013895747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:38.354352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:38.356841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:38.362206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 887/1767] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:37 2022] DELETE /identity/v3/projects/2d78139f10f041bbab2fc9e013895747 => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:38.367812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:38.370012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:38.503906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e26fe-601c-4ed0-9f4a-47b55594ed28 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:38.518045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f966dc5-c5ed-41a9-8d48-418538495be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:38.518723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f966dc5-c5ed-41a9-8d48-418538495be2 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:38.519010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f966dc5-c5ed-41a9-8d48-418538495be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:38.519952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f966dc5-c5ed-41a9-8d48-418538495be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:38.520078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f966dc5-c5ed-41a9-8d48-418538495be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:38.529902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f966dc5-c5ed-41a9-8d48-418538495be2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:38.537317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88510ede-a02d-40c5-b63d-32690243bb09 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:38.550678 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:38.553502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:38.555072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:38.555485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] PATH_INFO: `/v3/users/af705f571d09432b9044c9f5f9e9ca27` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:38.559543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af705f571d09432b9044c9f5f9e9ca27)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:38.564710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:38.598883 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 881/1768] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:38 2022] POST /identity/v3/projects => generated 373 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:38.602950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f966dc5-c5ed-41a9-8d48-418538495be2 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:38.604693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9f966dc5-c5ed-41a9-8d48-418538495be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:38.719185 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af705f571d09432b9044c9f5f9e9ca27'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:38.738490 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af705f571d09432b9044c9f5f9e9ca27'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:38.739856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af705f571d09432b9044c9f5f9e9ca27'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:38.761335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af705f571d09432b9044c9f5f9e9ca27'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:38.761753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af705f571d09432b9044c9f5f9e9ca27'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:38.816942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:38.819893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 888/1769] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:38 2022] DELETE /identity/v3/users/af705f571d09432b9044c9f5f9e9ca27 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:07:38.832690 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f966dc5-c5ed-41a9-8d48-418538495be2 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:38.992358 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07deda9b-5beb-430e-b275-519272b723a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:38.992943 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deda9b-5beb-430e-b275-519272b723a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:38.993233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deda9b-5beb-430e-b275-519272b723a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:38.993531 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deda9b-5beb-430e-b275-519272b723a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:38.994222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07deda9b-5beb-430e-b275-519272b723a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:39.004396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07deda9b-5beb-430e-b275-519272b723a6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:39.073125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:39.075211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:39.076588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 882/1770] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:38 2022] POST /identity/v3/users => generated 293 bytes in 473 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:07:39.113246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750ec49-dfa3-4016-9448-fb9b7af9d047 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:39.134731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaa462-b882-4da9-939b-9ffe06b93207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:39.135542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaa462-b882-4da9-939b-9ffe06b93207 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:39.135832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaa462-b882-4da9-939b-9ffe06b93207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:39.136670 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaa462-b882-4da9-939b-9ffe06b93207 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:39.137485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aaa462-b882-4da9-939b-9ffe06b93207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:39.142931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aaa462-b882-4da9-939b-9ffe06b93207 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:39.148475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:39.171827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 889/1771] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:07:39.173695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07deda9b-5beb-430e-b275-519272b723a6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:39.192714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07deda9b-5beb-430e-b275-519272b723a6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:39.200249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7decf-9e08-418d-98c0-00bbb355a110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:39.201594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7decf-9e08-418d-98c0-00bbb355a110 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:39.202097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7decf-9e08-418d-98c0-00bbb355a110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:39.202581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7decf-9e08-418d-98c0-00bbb355a110 admin admin] PATH_INFO: `/v3/projects/c5e987f9bcbe4d5db246a41b8d98f09f/users/f7f073d54b7b47949cd5125a069971de/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:39.231626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7decf-9e08-418d-98c0-00bbb355a110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e987f9bcbe4d5db246a41b8d98f09f, user_id=f7f073d54b7b47949cd5125a069971de, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:39.238865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7decf-9e08-418d-98c0-00bbb355a110 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:39.274729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 883/1772] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:39 2022] PUT /identity/v3/projects/c5e987f9bcbe4d5db246a41b8d98f09f/users/f7f073d54b7b47949cd5125a069971de/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:39.279772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaa462-b882-4da9-939b-9ffe06b93207 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:39.300181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaa462-b882-4da9-939b-9ffe06b93207 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:39.482340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77c501-5f34-4785-a302-e1ce67e1ed4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:39.484446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77c501-5f34-4785-a302-e1ce67e1ed4f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:39.485656 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77c501-5f34-4785-a302-e1ce67e1ed4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:39.486716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77c501-5f34-4785-a302-e1ce67e1ed4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:39.489909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77c501-5f34-4785-a302-e1ce67e1ed4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:39.491918 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77c501-5f34-4785-a302-e1ce67e1ed4f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:39.504935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 890/1773] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:39.509995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7decf-9e08-418d-98c0-00bbb355a110 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:39.522276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7decf-9e08-418d-98c0-00bbb355a110 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:39.540854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12ce28-51ff-4c3a-8560-efea624f1b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:39.542269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12ce28-51ff-4c3a-8560-efea624f1b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:39.542790 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12ce28-51ff-4c3a-8560-efea624f1b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:39.543309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12ce28-51ff-4c3a-8560-efea624f1b12 admin admin] PATH_INFO: `/v3/projects/c5e987f9bcbe4d5db246a41b8d98f09f/users/f7f073d54b7b47949cd5125a069971de/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:39.545993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12ce28-51ff-4c3a-8560-efea624f1b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e987f9bcbe4d5db246a41b8d98f09f, user_id=f7f073d54b7b47949cd5125a069971de, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:39.549204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12ce28-51ff-4c3a-8560-efea624f1b12 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:39.570249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 884/1774] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:39 2022] PUT /identity/v3/projects/c5e987f9bcbe4d5db246a41b8d98f09f/users/f7f073d54b7b47949cd5125a069971de/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:39.585494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0441ac1-bd07-47ee-978d-58e2e0c66028 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:39.586064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0441ac1-bd07-47ee-978d-58e2e0c66028 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:39.586360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0441ac1-bd07-47ee-978d-58e2e0c66028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:39.630834 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b0441ac1-bd07-47ee-978d-58e2e0c66028 None None] MFA Rules not processed for user `f7f073d54b7b47949cd5125a069971de`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:39.632020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b0441ac1-bd07-47ee-978d-58e2e0c66028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:39.738657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 891/1775] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:39.749807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12ce28-51ff-4c3a-8560-efea624f1b12 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:39.753101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1e12ce28-51ff-4c3a-8560-efea624f1b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:39.849511 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12ce28-51ff-4c3a-8560-efea624f1b12 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:39.857200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6cdb5-73f7-403a-8f5d-8c83afc35fd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:39.858364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6cdb5-73f7-403a-8f5d-8c83afc35fd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:39.858880 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6cdb5-73f7-403a-8f5d-8c83afc35fd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:39.863416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6cdb5-73f7-403a-8f5d-8c83afc35fd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:39.880194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6cdb5-73f7-403a-8f5d-8c83afc35fd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:39.881755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6cdb5-73f7-403a-8f5d-8c83afc35fd1 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:39.903610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0441ac1-bd07-47ee-978d-58e2e0c66028 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:39.905400 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b0441ac1-bd07-47ee-978d-58e2e0c66028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:39.920559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 885/1776] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:40.079601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6cdb5-73f7-403a-8f5d-8c83afc35fd1 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:40.092281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6cdb5-73f7-403a-8f5d-8c83afc35fd1 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:40.104160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16040f3-c1be-414e-86c1-f8ba7978537e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:40.105364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16040f3-c1be-414e-86c1-f8ba7978537e service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:40.105669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16040f3-c1be-414e-86c1-f8ba7978537e service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:40.105941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16040f3-c1be-414e-86c1-f8ba7978537e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:40.114352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c16040f3-c1be-414e-86c1-f8ba7978537e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:40.157819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0441ac1-bd07-47ee-978d-58e2e0c66028 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:40.177244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1138039-e611-4428-ac8a-e2b459c0bd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:40.178008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1138039-e611-4428-ac8a-e2b459c0bd7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:40.178292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1138039-e611-4428-ac8a-e2b459c0bd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:40.178575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1138039-e611-4428-ac8a-e2b459c0bd7e admin admin] PATH_INFO: `/v3/projects/8e42462377304596979258660371bdbf` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:40.180381 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1138039-e611-4428-ac8a-e2b459c0bd7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e42462377304596979258660371bdbf)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:40.182952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1138039-e611-4428-ac8a-e2b459c0bd7e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:40.299020 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:40.300686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:40.304324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16040f3-c1be-414e-86c1-f8ba7978537e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:40.308018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16040f3-c1be-414e-86c1-f8ba7978537e service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:40.334851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 886/1777] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:07:40.353343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b1138039-e611-4428-ac8a-e2b459c0bd7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e42462377304596979258660371bdbf', 'request_id': 'req-b1138039-e611-4428-ac8a-e2b459c0bd7e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:40.393880 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b1138039-e611-4428-ac8a-e2b459c0bd7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e42462377304596979258660371bdbf', 'request_id': 'req-b1138039-e611-4428-ac8a-e2b459c0bd7e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:40.409955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b1138039-e611-4428-ac8a-e2b459c0bd7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e42462377304596979258660371bdbf', 'request_id': 'req-b1138039-e611-4428-ac8a-e2b459c0bd7e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:40.433123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b1138039-e611-4428-ac8a-e2b459c0bd7e admin admin] The token cache is being invalidate because project 8e42462377304596979258660371bdbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:40.433487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b1138039-e611-4428-ac8a-e2b459c0bd7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:40.434763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-b1138039-e611-4428-ac8a-e2b459c0bd7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:40.438538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 892/1778] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:39 2022] DELETE /identity/v3/projects/8e42462377304596979258660371bdbf => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:40.443829 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2c032-8c19-4ec9-9571-03a40c324227 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:40.444118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2c032-8c19-4ec9-9571-03a40c324227 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:40.444582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2c032-8c19-4ec9-9571-03a40c324227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:40.482243 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0df2c032-8c19-4ec9-9571-03a40c324227 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:40.485120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0df2c032-8c19-4ec9-9571-03a40c324227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:40.649469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 887/1779] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:40.659110 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592434c-e0d1-408c-8a3c-ee3adb44c68d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:40.659435 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592434c-e0d1-408c-8a3c-ee3adb44c68d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:40.659780 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592434c-e0d1-408c-8a3c-ee3adb44c68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:40.744815 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a592434c-e0d1-408c-8a3c-ee3adb44c68d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:40.746979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a592434c-e0d1-408c-8a3c-ee3adb44c68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:40.785205 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 893/1780] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:40.785821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2c032-8c19-4ec9-9571-03a40c324227 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:40.829903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2c032-8c19-4ec9-9571-03a40c324227 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:40.849266 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ddb56-4d8d-4964-a88f-e86e8a6193da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:40.850064 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ddb56-4d8d-4964-a88f-e86e8a6193da admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:40.850395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ddb56-4d8d-4964-a88f-e86e8a6193da admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:40.850727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ddb56-4d8d-4964-a88f-e86e8a6193da admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:40.855123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ddb56-4d8d-4964-a88f-e86e8a6193da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:40.856035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ddb56-4d8d-4964-a88f-e86e8a6193da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:40.864241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ddb56-4d8d-4964-a88f-e86e8a6193da admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:40.905357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 888/1781] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:07:40.906272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592434c-e0d1-408c-8a3c-ee3adb44c68d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:40.916564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592434c-e0d1-408c-8a3c-ee3adb44c68d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:40.977257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c05ce0-892c-430a-aeb4-1b18d5b382a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:40.980769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c05ce0-892c-430a-aeb4-1b18d5b382a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:40.981304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c05ce0-892c-430a-aeb4-1b18d5b382a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:40.981767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c05ce0-892c-430a-aeb4-1b18d5b382a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:40.985046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c05ce0-892c-430a-aeb4-1b18d5b382a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:40.994028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c05ce0-892c-430a-aeb4-1b18d5b382a8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:41.099004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 894/1782] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:40 2022] POST /identity/v3/projects => generated 367 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:07:41.102954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ddb56-4d8d-4964-a88f-e86e8a6193da admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:41.126066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ddb56-4d8d-4964-a88f-e86e8a6193da admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:41.425844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7483d5a-124d-40d1-a965-b58ed5518446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:41.428318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7483d5a-124d-40d1-a965-b58ed5518446 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:41.429004 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7483d5a-124d-40d1-a965-b58ed5518446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:41.429972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7483d5a-124d-40d1-a965-b58ed5518446 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:41.432901 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7483d5a-124d-40d1-a965-b58ed5518446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:41.441449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7483d5a-124d-40d1-a965-b58ed5518446 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:41.541764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 889/1783] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:41 2022] POST /identity/v3/users => generated 284 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:41.543867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c05ce0-892c-430a-aeb4-1b18d5b382a8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:41.572002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c05ce0-892c-430a-aeb4-1b18d5b382a8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:41.593953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6a876-52ca-41dc-807b-8750d9d6a337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:41.594664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6a876-52ca-41dc-807b-8750d9d6a337 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:41.594947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6a876-52ca-41dc-807b-8750d9d6a337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:41.595230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6a876-52ca-41dc-807b-8750d9d6a337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:41.597090 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6a876-52ca-41dc-807b-8750d9d6a337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:41.599270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6a876-52ca-41dc-807b-8750d9d6a337 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:41.648512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 895/1784] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:41.654898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7483d5a-124d-40d1-a965-b58ed5518446 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:41.687819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7483d5a-124d-40d1-a965-b58ed5518446 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:41.712445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddb44d-d2e1-49f4-be6c-d621865e13b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:41.717961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddb44d-d2e1-49f4-be6c-d621865e13b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:41.722831 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddb44d-d2e1-49f4-be6c-d621865e13b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:41.725688 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddb44d-d2e1-49f4-be6c-d621865e13b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:41.733974 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddb44d-d2e1-49f4-be6c-d621865e13b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:41.749795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddb44d-d2e1-49f4-be6c-d621865e13b8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:41.848655 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6a876-52ca-41dc-807b-8750d9d6a337 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:41.852093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 890/1785] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:41.865469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6a876-52ca-41dc-807b-8750d9d6a337 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:41.880784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54b3fb-8e81-40f0-bf9d-59b7238337a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:41.881812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54b3fb-8e81-40f0-bf9d-59b7238337a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:41.886485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54b3fb-8e81-40f0-bf9d-59b7238337a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:41.887491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54b3fb-8e81-40f0-bf9d-59b7238337a9 admin admin] PATH_INFO: `/v3/projects/ed7e3f280cb643d3be3730f787ffadff/users/a867288b830d479f9e4429b8a6ddfb8a/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:41.898279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f25588-bd5e-45f6-8879-94b2cd0a771f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:41.898876 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f25588-bd5e-45f6-8879-94b2cd0a771f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:41.899310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f25588-bd5e-45f6-8879-94b2cd0a771f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:41.924828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54b3fb-8e81-40f0-bf9d-59b7238337a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7e3f280cb643d3be3730f787ffadff, user_id=a867288b830d479f9e4429b8a6ddfb8a, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:41.929109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54b3fb-8e81-40f0-bf9d-59b7238337a9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:41.954996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 896/1786] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:41 2022] PUT /identity/v3/projects/ed7e3f280cb643d3be3730f787ffadff/users/a867288b830d479f9e4429b8a6ddfb8a/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:41.971713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a675-31c3-40e7-a8ed-e9d45b18e2d4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:41.972219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a675-31c3-40e7-a8ed-e9d45b18e2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:41.972710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a675-31c3-40e7-a8ed-e9d45b18e2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:42.046316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e2f25588-bd5e-45f6-8879-94b2cd0a771f None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:42.047887 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e2f25588-bd5e-45f6-8879-94b2cd0a771f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:42.100290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-64f2a675-31c3-40e7-a8ed-e9d45b18e2d4 None None] MFA Rules not processed for user `a867288b830d479f9e4429b8a6ddfb8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:42.105009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-64f2a675-31c3-40e7-a8ed-e9d45b18e2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:42.126139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:42.252707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 891/1787] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:07:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:42.421455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 897/1788] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:42.426893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f25588-bd5e-45f6-8879-94b2cd0a771f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:42.633169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2a675-31c3-40e7-a8ed-e9d45b18e2d4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:42.634964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-64f2a675-31c3-40e7-a8ed-e9d45b18e2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:42.661761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f25588-bd5e-45f6-8879-94b2cd0a771f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:42.670005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf83d57-ccbe-4e2b-9cf4-8972eb405608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:42.670712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf83d57-ccbe-4e2b-9cf4-8972eb405608 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:42.671009 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf83d57-ccbe-4e2b-9cf4-8972eb405608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:42.671285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf83d57-ccbe-4e2b-9cf4-8972eb405608 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:42.672066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf83d57-ccbe-4e2b-9cf4-8972eb405608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:42.674118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf83d57-ccbe-4e2b-9cf4-8972eb405608 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:42.732825 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 892/1789] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:42 2022] POST /identity/v3/projects => generated 369 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:42.735329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf83d57-ccbe-4e2b-9cf4-8972eb405608 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:42.816714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:42.992583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf83d57-ccbe-4e2b-9cf4-8972eb405608 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:42.999374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc312224-5a4f-4f8d-bfe2-072161db4279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.000109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc312224-5a4f-4f8d-bfe2-072161db4279 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.000401 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc312224-5a4f-4f8d-bfe2-072161db4279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.000748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc312224-5a4f-4f8d-bfe2-072161db4279 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.001464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc312224-5a4f-4f8d-bfe2-072161db4279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.004408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc312224-5a4f-4f8d-bfe2-072161db4279 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:43.055264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 893/1790] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:42 2022] POST /identity/v3/users => generated 291 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:43.060708 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc312224-5a4f-4f8d-bfe2-072161db4279 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:43.098744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc312224-5a4f-4f8d-bfe2-072161db4279 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:43.114229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731dea4-8aa9-4e66-a302-7ea6a4fb3138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.117687 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2a675-31c3-40e7-a8ed-e9d45b18e2d4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:43.117912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731dea4-8aa9-4e66-a302-7ea6a4fb3138 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.118220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731dea4-8aa9-4e66-a302-7ea6a4fb3138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.118574 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731dea4-8aa9-4e66-a302-7ea6a4fb3138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.121722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731dea4-8aa9-4e66-a302-7ea6a4fb3138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.127108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731dea4-8aa9-4e66-a302-7ea6a4fb3138 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:43.156642 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 894/1791] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:43.163312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731dea4-8aa9-4e66-a302-7ea6a4fb3138 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:43.192931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731dea4-8aa9-4e66-a302-7ea6a4fb3138 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:43.231584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01160fcd-df77-445a-bb86-3ea3ca1f7b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.233149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160fcd-df77-445a-bb86-3ea3ca1f7b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.243005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160fcd-df77-445a-bb86-3ea3ca1f7b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.243306 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160fcd-df77-445a-bb86-3ea3ca1f7b52 admin admin] PATH_INFO: `/v3/projects/261d958c5a8847178473cab5b663d024/users/2567d96a51234ce8892251dab738e9f7/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.337564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01160fcd-df77-445a-bb86-3ea3ca1f7b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=261d958c5a8847178473cab5b663d024, user_id=2567d96a51234ce8892251dab738e9f7, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.350315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01160fcd-df77-445a-bb86-3ea3ca1f7b52 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:43.410435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 895/1792] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:43 2022] PUT /identity/v3/projects/261d958c5a8847178473cab5b663d024/users/2567d96a51234ce8892251dab738e9f7/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:43.411950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01160fcd-df77-445a-bb86-3ea3ca1f7b52 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:43.457732 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76931fea-4600-485b-b3e4-167216c16003 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.458475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76931fea-4600-485b-b3e4-167216c16003 service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.462579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76931fea-4600-485b-b3e4-167216c16003 service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.465511 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76931fea-4600-485b-b3e4-167216c16003 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.481990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:43.760489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01160fcd-df77-445a-bb86-3ea3ca1f7b52 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:43.770881 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76931fea-4600-485b-b3e4-167216c16003 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.778181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76931fea-4600-485b-b3e4-167216c16003 service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:43.786788 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e89b45-5a38-4e79-b315-d076670244a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.787533 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e89b45-5a38-4e79-b315-d076670244a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.787821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e89b45-5a38-4e79-b315-d076670244a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.788097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e89b45-5a38-4e79-b315-d076670244a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.788844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e89b45-5a38-4e79-b315-d076670244a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.790480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e89b45-5a38-4e79-b315-d076670244a0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:43.831487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 898/1793] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:07:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 11:07:43.858953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76931fea-4600-485b-b3e4-167216c16003 service swift] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:43.873491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 896/1794] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:43.889433 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76931fea-4600-485b-b3e4-167216c16003 service swift] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:43.908913 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67533f-68aa-4a4b-b232-cfaadcd2b5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.910570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67533f-68aa-4a4b-b232-cfaadcd2b5ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.911125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67533f-68aa-4a4b-b232-cfaadcd2b5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.911439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67533f-68aa-4a4b-b232-cfaadcd2b5ec admin admin] PATH_INFO: `/v3/projects/261d958c5a8847178473cab5b663d024/users/2567d96a51234ce8892251dab738e9f7/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.927177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67533f-68aa-4a4b-b232-cfaadcd2b5ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=261d958c5a8847178473cab5b663d024, user_id=2567d96a51234ce8892251dab738e9f7, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.928908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67533f-68aa-4a4b-b232-cfaadcd2b5ec admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:44.002870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 899/1795] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:43 2022] PUT /identity/v3/projects/261d958c5a8847178473cab5b663d024/users/2567d96a51234ce8892251dab738e9f7/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:07:44.016781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e870c4-b8ab-40e2-bbad-74a534049ddd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:44.017054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e870c4-b8ab-40e2-bbad-74a534049ddd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:44.017355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e870c4-b8ab-40e2-bbad-74a534049ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:44.101692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b4e870c4-b8ab-40e2-bbad-74a534049ddd None None] MFA Rules not processed for user `2567d96a51234ce8892251dab738e9f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:44.108231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b4e870c4-b8ab-40e2-bbad-74a534049ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:44.455788 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 897/1796] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:44 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:44.700562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67533f-68aa-4a4b-b232-cfaadcd2b5ec admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:44.702470 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3a67533f-68aa-4a4b-b232-cfaadcd2b5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:44.903461 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:44.940736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67533f-68aa-4a4b-b232-cfaadcd2b5ec admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:44.971245 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3572947-966e-4f63-9661-a04e56244546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:44.974040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3572947-966e-4f63-9661-a04e56244546 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:44.974352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3572947-966e-4f63-9661-a04e56244546 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:44.974662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3572947-966e-4f63-9661-a04e56244546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:45.017516 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3572947-966e-4f63-9661-a04e56244546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:45.020606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3572947-966e-4f63-9661-a04e56244546 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:45.080975 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 900/1797] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:44 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:45.282686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e870c4-b8ab-40e2-bbad-74a534049ddd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:45.291040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b4e870c4-b8ab-40e2-bbad-74a534049ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:45.472154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3572947-966e-4f63-9661-a04e56244546 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:45.473703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a3572947-966e-4f63-9661-a04e56244546 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:45.518685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3572947-966e-4f63-9661-a04e56244546 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:45.554213 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97e941-07c2-4b82-b56c-4113553e4cc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:45.556082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97e941-07c2-4b82-b56c-4113553e4cc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:45.556082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97e941-07c2-4b82-b56c-4113553e4cc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:45.556082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97e941-07c2-4b82-b56c-4113553e4cc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:45.668807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e870c4-b8ab-40e2-bbad-74a534049ddd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:45.692772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:45.693700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:45.693990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:45.694273 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] PATH_INFO: `/v3/users/b6575d95c20846ab8c945cfb9515e5e8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:45.710888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6575d95c20846ab8c945cfb9515e5e8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:45.713521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:45.843700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97e941-07c2-4b82-b56c-4113553e4cc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:45.847875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97e941-07c2-4b82-b56c-4113553e4cc6 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:45.869044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:45.891494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 901/1798] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:07:45.923250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6575d95c20846ab8c945cfb9515e5e8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:45.924812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6575d95c20846ab8c945cfb9515e5e8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:45.959069 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6575d95c20846ab8c945cfb9515e5e8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:45.975450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6575d95c20846ab8c945cfb9515e5e8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:45.976461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6575d95c20846ab8c945cfb9515e5e8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:46.039259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 898/1799] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:45 2022] DELETE /identity/v3/users/b6575d95c20846ab8c945cfb9515e5e8 => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:46.045649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62be857-1ab6-4871-a769-c16dd07fa898 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:46.046167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62be857-1ab6-4871-a769-c16dd07fa898 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:46.046665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62be857-1ab6-4871-a769-c16dd07fa898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:46.115536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d62be857-1ab6-4871-a769-c16dd07fa898 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:46.116966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d62be857-1ab6-4871-a769-c16dd07fa898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:46.366803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 902/1800] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:46.431696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:46.434271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:46.598196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625f32f-3936-4f24-bbac-9d28c2a6da5c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:46.623911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d91bd-04c3-4937-b816-01ce47101912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:46.624654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d91bd-04c3-4937-b816-01ce47101912 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:46.624955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d91bd-04c3-4937-b816-01ce47101912 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:46.625289 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d91bd-04c3-4937-b816-01ce47101912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:46.673993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d91bd-04c3-4937-b816-01ce47101912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:46.675673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d91bd-04c3-4937-b816-01ce47101912 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:46.742259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 899/1801] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:47.425069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62be857-1ab6-4871-a769-c16dd07fa898 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:47.427123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d62be857-1ab6-4871-a769-c16dd07fa898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:47.447304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62be857-1ab6-4871-a769-c16dd07fa898 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:47.452752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:47.453731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:47.454272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:47.454685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] PATH_INFO: `/v3/projects/fa15e7e39a1746198bdc2be02bb5eee4` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:47.456006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa15e7e39a1746198bdc2be02bb5eee4)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:47.457966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:47.541718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa15e7e39a1746198bdc2be02bb5eee4', 'request_id': 'req-4dba884e-f119-4f9d-853a-589347ee61bf'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:47.550720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa15e7e39a1746198bdc2be02bb5eee4', 'request_id': 'req-4dba884e-f119-4f9d-853a-589347ee61bf'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:47.558956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa15e7e39a1746198bdc2be02bb5eee4', 'request_id': 'req-4dba884e-f119-4f9d-853a-589347ee61bf'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:47.574998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] The token cache is being invalidate because project fa15e7e39a1746198bdc2be02bb5eee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:47.575299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:47.576060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:47.577948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 903/1802] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:47 2022] DELETE /identity/v3/projects/fa15e7e39a1746198bdc2be02bb5eee4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:47.582345 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d91bd-04c3-4937-b816-01ce47101912 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:47.594061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d91bd-04c3-4937-b816-01ce47101912 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:47.706200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:47.707610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:47.708055 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:47.708516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] PATH_INFO: `/v3/users/e9b44b5c38294d4f81c849fd8a6aa708` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:47.710985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9b44b5c38294d4f81c849fd8a6aa708)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:47.713740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:47.786305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9b44b5c38294d4f81c849fd8a6aa708'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:47.787152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9b44b5c38294d4f81c849fd8a6aa708'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:47.797214 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9b44b5c38294d4f81c849fd8a6aa708'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:47.799203 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:47.807333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9b44b5c38294d4f81c849fd8a6aa708'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:47.808037 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9b44b5c38294d4f81c849fd8a6aa708'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:47.835488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 900/1803] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:47 2022] DELETE /identity/v3/users/e9b44b5c38294d4f81c849fd8a6aa708 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:48.237336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:48.239407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:48.280893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba884e-f119-4f9d-853a-589347ee61bf admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:48.294795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:48.368925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:48.371066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:48.371456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:48.371964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] PATH_INFO: `/v3/projects/f380151590e645328c138113e7d13081` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:48.373572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f380151590e645328c138113e7d13081)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:48.376562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:48.468695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f380151590e645328c138113e7d13081', 'request_id': 'req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:48.475998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f380151590e645328c138113e7d13081', 'request_id': 'req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:48.482572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f380151590e645328c138113e7d13081', 'request_id': 'req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:48.501304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] The token cache is being invalidate because project f380151590e645328c138113e7d13081 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:48.501606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:48.502276 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:48.504148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 904/1804] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:48 2022] DELETE /identity/v3/projects/f380151590e645328c138113e7d13081 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:48.508447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:48.510032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:48.524951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7367a3-1338-4073-b111-6f8fc1c7254e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:48.599752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c16a2e-c9dc-4a81-8be6-e803055767b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:48.600696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c16a2e-c9dc-4a81-8be6-e803055767b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:48.601009 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c16a2e-c9dc-4a81-8be6-e803055767b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:48.601319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c16a2e-c9dc-4a81-8be6-e803055767b1 admin admin] PATH_INFO: `/v3/users/d3eee652f5c64fd4bf7e7c0d3af1a88e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:48.602574 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c16a2e-c9dc-4a81-8be6-e803055767b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3eee652f5c64fd4bf7e7c0d3af1a88e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:48.604670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c16a2e-c9dc-4a81-8be6-e803055767b1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:48.667793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-08c16a2e-c9dc-4a81-8be6-e803055767b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3eee652f5c64fd4bf7e7c0d3af1a88e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:48.668533 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-08c16a2e-c9dc-4a81-8be6-e803055767b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3eee652f5c64fd4bf7e7c0d3af1a88e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:48.681630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-08c16a2e-c9dc-4a81-8be6-e803055767b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3eee652f5c64fd4bf7e7c0d3af1a88e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:48.693389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-08c16a2e-c9dc-4a81-8be6-e803055767b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3eee652f5c64fd4bf7e7c0d3af1a88e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:48.694033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-08c16a2e-c9dc-4a81-8be6-e803055767b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3eee652f5c64fd4bf7e7c0d3af1a88e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:48.718022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 901/1805] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:48 2022] DELETE /identity/v3/users/d3eee652f5c64fd4bf7e7c0d3af1a88e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:07:49.135092 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:49.136702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:49.153569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae4b0a-abfa-4ed1-9a4a-086fb52ec6d8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:49.225801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87029201-0e1b-4da3-a03e-6578c0e21e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:49.226782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87029201-0e1b-4da3-a03e-6578c0e21e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:49.227129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87029201-0e1b-4da3-a03e-6578c0e21e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:49.227435 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87029201-0e1b-4da3-a03e-6578c0e21e42 admin admin] PATH_INFO: `/v3/projects/3947ee4dc32e4d7c85b62110269c71f2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:49.228756 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87029201-0e1b-4da3-a03e-6578c0e21e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3947ee4dc32e4d7c85b62110269c71f2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:49.231762 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87029201-0e1b-4da3-a03e-6578c0e21e42 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:49.284901 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:49.289270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-87029201-0e1b-4da3-a03e-6578c0e21e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3947ee4dc32e4d7c85b62110269c71f2', 'request_id': 'req-87029201-0e1b-4da3-a03e-6578c0e21e42'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:49.298143 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-87029201-0e1b-4da3-a03e-6578c0e21e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3947ee4dc32e4d7c85b62110269c71f2', 'request_id': 'req-87029201-0e1b-4da3-a03e-6578c0e21e42'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:49.306552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-87029201-0e1b-4da3-a03e-6578c0e21e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3947ee4dc32e4d7c85b62110269c71f2', 'request_id': 'req-87029201-0e1b-4da3-a03e-6578c0e21e42'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:49.321127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-87029201-0e1b-4da3-a03e-6578c0e21e42 admin admin] The token cache is being invalidate because project 3947ee4dc32e4d7c85b62110269c71f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:49.321398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-87029201-0e1b-4da3-a03e-6578c0e21e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:49.322069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-87029201-0e1b-4da3-a03e-6578c0e21e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:49.323705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 905/1806] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:49 2022] DELETE /identity/v3/projects/3947ee4dc32e4d7c85b62110269c71f2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:49.330555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81313a2-4606-47c5-ab7c-bf70c141eb46 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:49.331022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81313a2-4606-47c5-ab7c-bf70c141eb46 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:49.331487 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81313a2-4606-47c5-ab7c-bf70c141eb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:49.363968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e81313a2-4606-47c5-ab7c-bf70c141eb46 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:49.365393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e81313a2-4606-47c5-ab7c-bf70c141eb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:49.431118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:49.452115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 902/1807] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:49.461468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432448f1-ce10-4cff-b923-23829623e7c3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:49.461776 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432448f1-ce10-4cff-b923-23829623e7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:49.462066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432448f1-ce10-4cff-b923-23829623e7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:49.485953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-432448f1-ce10-4cff-b923-23829623e7c3 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:49.486487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-432448f1-ce10-4cff-b923-23829623e7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:49.517032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 906/1808] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:49.518403 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81313a2-4606-47c5-ab7c-bf70c141eb46 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:49.531573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81313a2-4606-47c5-ab7c-bf70c141eb46 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:49.536800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b1d1a-91da-47f2-8eff-05cd42eeeb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:49.537472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b1d1a-91da-47f2-8eff-05cd42eeeb8f admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:49.537758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b1d1a-91da-47f2-8eff-05cd42eeeb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:49.538035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b1d1a-91da-47f2-8eff-05cd42eeeb8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:49.538782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b1d1a-91da-47f2-8eff-05cd42eeeb8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:49.539134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b1d1a-91da-47f2-8eff-05cd42eeeb8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:49.540914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b1d1a-91da-47f2-8eff-05cd42eeeb8f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:49.555901 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 903/1809] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:49.561156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432448f1-ce10-4cff-b923-23829623e7c3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:49.572201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432448f1-ce10-4cff-b923-23829623e7c3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:49.578059 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e4b5a-d25a-4a4a-ab40-93f92d2f0bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:49.578930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e4b5a-d25a-4a4a-ab40-93f92d2f0bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:49.579341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e4b5a-d25a-4a4a-ab40-93f92d2f0bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:49.579821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e4b5a-d25a-4a4a-ab40-93f92d2f0bd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:49.580962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e4b5a-d25a-4a4a-ab40-93f92d2f0bd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:49.585527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e4b5a-d25a-4a4a-ab40-93f92d2f0bd4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:49.628903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 907/1810] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:49 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:49.633616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b1d1a-91da-47f2-8eff-05cd42eeeb8f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:49.646693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b1d1a-91da-47f2-8eff-05cd42eeeb8f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:49.738759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb92d4a-f530-4dc9-9b7d-cebb2dec26ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:49.739779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb92d4a-f530-4dc9-9b7d-cebb2dec26ba admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:49.740160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb92d4a-f530-4dc9-9b7d-cebb2dec26ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:49.740573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb92d4a-f530-4dc9-9b7d-cebb2dec26ba admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:49.741396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb92d4a-f530-4dc9-9b7d-cebb2dec26ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:49.743633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb92d4a-f530-4dc9-9b7d-cebb2dec26ba admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:49.780264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 904/1811] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:49 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:07:49.784840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e4b5a-d25a-4a4a-ab40-93f92d2f0bd4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:49.798649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e4b5a-d25a-4a4a-ab40-93f92d2f0bd4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:49.819646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63671ce6-fc91-4f82-8d3d-998538b98d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:49.820864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63671ce6-fc91-4f82-8d3d-998538b98d70 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:49.821448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63671ce6-fc91-4f82-8d3d-998538b98d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:49.821966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63671ce6-fc91-4f82-8d3d-998538b98d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:49.823034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63671ce6-fc91-4f82-8d3d-998538b98d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:49.826970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63671ce6-fc91-4f82-8d3d-998538b98d70 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:49.851231 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 908/1812] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:49.851713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb92d4a-f530-4dc9-9b7d-cebb2dec26ba admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:49.874175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb92d4a-f530-4dc9-9b7d-cebb2dec26ba admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:49.899248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b6ddc-c0c4-4045-814e-1802babeff96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:49.903597 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b6ddc-c0c4-4045-814e-1802babeff96 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:49.903898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b6ddc-c0c4-4045-814e-1802babeff96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:49.904251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b6ddc-c0c4-4045-814e-1802babeff96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:49.907101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b6ddc-c0c4-4045-814e-1802babeff96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:49.912828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b6ddc-c0c4-4045-814e-1802babeff96 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:49.955951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 905/1813] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:49.964055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63671ce6-fc91-4f82-8d3d-998538b98d70 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:49.977228 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63671ce6-fc91-4f82-8d3d-998538b98d70 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:49.983508 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15013e30-6ae6-43da-abec-713e76f27d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:49.985266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15013e30-6ae6-43da-abec-713e76f27d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:49.985779 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15013e30-6ae6-43da-abec-713e76f27d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:49.986236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15013e30-6ae6-43da-abec-713e76f27d33 admin admin] PATH_INFO: `/v3/projects/0d4bd52b13e04ef9a7143d5b3f352339/users/1623525da690405c9c4bcf5e66eb122d/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:50.037411 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15013e30-6ae6-43da-abec-713e76f27d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4bd52b13e04ef9a7143d5b3f352339, user_id=1623525da690405c9c4bcf5e66eb122d, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:50.042068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15013e30-6ae6-43da-abec-713e76f27d33 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:50.075131 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 909/1814] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:49 2022] PUT /identity/v3/projects/0d4bd52b13e04ef9a7143d5b3f352339/users/1623525da690405c9c4bcf5e66eb122d/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:50.098485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0e48d-c1f1-4739-9401-40ec948ac050 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:50.099127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0e48d-c1f1-4739-9401-40ec948ac050 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:50.100066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0e48d-c1f1-4739-9401-40ec948ac050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:50.165444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-7db0e48d-c1f1-4739-9401-40ec948ac050 None None] MFA Rules not processed for user `1623525da690405c9c4bcf5e66eb122d`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:50.168234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7db0e48d-c1f1-4739-9401-40ec948ac050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:50.321871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 906/1815] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:50.326491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15013e30-6ae6-43da-abec-713e76f27d33 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:50.454348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d192a7-9fc5-4d85-badf-7c28a56a295c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:50.454348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d192a7-9fc5-4d85-badf-7c28a56a295c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:50.454348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d192a7-9fc5-4d85-badf-7c28a56a295c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:50.504395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15013e30-6ae6-43da-abec-713e76f27d33 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:50.514371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690aa1e4-3e34-46da-b82a-d2e84efff190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:50.515591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690aa1e4-3e34-46da-b82a-d2e84efff190 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:50.516206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690aa1e4-3e34-46da-b82a-d2e84efff190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:50.516649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690aa1e4-3e34-46da-b82a-d2e84efff190 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:50.517520 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690aa1e4-3e34-46da-b82a-d2e84efff190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:50.520302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690aa1e4-3e34-46da-b82a-d2e84efff190 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:50.532173 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-41d192a7-9fc5-4d85-badf-7c28a56a295c None None] MFA Rules not processed for user `dadd485b7cac42f4aa079cab91973c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:50.537991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-41d192a7-9fc5-4d85-badf-7c28a56a295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:50.584088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 910/1816] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:50 2022] POST /identity/v3/projects => generated 379 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:50.588286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690aa1e4-3e34-46da-b82a-d2e84efff190 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:50.775224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:50.805143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 907/1817] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Jul 14 11:07:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:50.808394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d192a7-9fc5-4d85-badf-7c28a56a295c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:50.818352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-41d192a7-9fc5-4d85-badf-7c28a56a295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:50.839366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:50.885394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d192a7-9fc5-4d85-badf-7c28a56a295c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:50.892000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690aa1e4-3e34-46da-b82a-d2e84efff190 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:50.901464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:50.902464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:50.902761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:50.903056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] PATH_INFO: `/v3/users/a867288b830d479f9e4429b8a6ddfb8a` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:50.905093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a867288b830d479f9e4429b8a6ddfb8a)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:50.908833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807ec2a-1620-423a-bb9d-610fa72703ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:50.909587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807ec2a-1620-423a-bb9d-610fa72703ce admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:50.909969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807ec2a-1620-423a-bb9d-610fa72703ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:50.910254 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807ec2a-1620-423a-bb9d-610fa72703ce admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:50.910967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807ec2a-1620-423a-bb9d-610fa72703ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:50.913095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807ec2a-1620-423a-bb9d-610fa72703ce admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:50.918038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:50.974853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 911/1818] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:50 2022] POST /identity/v3/users => generated 296 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:50.974853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807ec2a-1620-423a-bb9d-610fa72703ce admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:50.989389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8807ec2a-1620-423a-bb9d-610fa72703ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:51.067308 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a867288b830d479f9e4429b8a6ddfb8a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:51.070406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a867288b830d479f9e4429b8a6ddfb8a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:51.096232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a867288b830d479f9e4429b8a6ddfb8a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:51.117862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a867288b830d479f9e4429b8a6ddfb8a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:51.119021 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a867288b830d479f9e4429b8a6ddfb8a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:51.198980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 908/1819] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:50 2022] DELETE /identity/v3/users/a867288b830d479f9e4429b8a6ddfb8a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:51.200521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:51.206102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:51.230885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807ec2a-1620-423a-bb9d-610fa72703ce admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:51.422993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762f5a5-66b7-45e6-920d-3d1c2cb8b73c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:51.423969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762f5a5-66b7-45e6-920d-3d1c2cb8b73c service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:51.424887 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762f5a5-66b7-45e6-920d-3d1c2cb8b73c service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:51.425597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762f5a5-66b7-45e6-920d-3d1c2cb8b73c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:51.437828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d762f5a5-66b7-45e6-920d-3d1c2cb8b73c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:51.469352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762f5a5-66b7-45e6-920d-3d1c2cb8b73c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:51.471922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762f5a5-66b7-45e6-920d-3d1c2cb8b73c service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:51.491580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 912/1820] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:51.492489 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762f5a5-66b7-45e6-920d-3d1c2cb8b73c service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:51.501253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d762f5a5-66b7-45e6-920d-3d1c2cb8b73c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:51.533952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762f5a5-66b7-45e6-920d-3d1c2cb8b73c service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:51.543239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bd4ed-7a5a-4e27-91dd-5c200a45e4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:51.544298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bd4ed-7a5a-4e27-91dd-5c200a45e4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:51.545759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bd4ed-7a5a-4e27-91dd-5c200a45e4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:51.546136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bd4ed-7a5a-4e27-91dd-5c200a45e4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:51.562084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94cc2a-9ba9-4667-8ae7-3a94cfcd7276 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:51.567034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bd4ed-7a5a-4e27-91dd-5c200a45e4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:51.569174 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bd4ed-7a5a-4e27-91dd-5c200a45e4c3 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:51.584743 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab17567-ec97-49ee-a5fd-ed7a0840d8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:51.585497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab17567-ec97-49ee-a5fd-ed7a0840d8fc admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:51.585867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab17567-ec97-49ee-a5fd-ed7a0840d8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:51.586263 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab17567-ec97-49ee-a5fd-ed7a0840d8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:51.588673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab17567-ec97-49ee-a5fd-ed7a0840d8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:51.590900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab17567-ec97-49ee-a5fd-ed7a0840d8fc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:51.602685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 913/1821] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:51.604101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d44b7a-5601-43d8-9872-956cebcec888 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:51.604578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d44b7a-5601-43d8-9872-956cebcec888 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:51.604979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d44b7a-5601-43d8-9872-956cebcec888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:51.642194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 909/1822] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:51.650413 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b4d44b7a-5601-43d8-9872-956cebcec888 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:51.652053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b4d44b7a-5601-43d8-9872-956cebcec888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:51.652782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab17567-ec97-49ee-a5fd-ed7a0840d8fc admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:51.671227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab17567-ec97-49ee-a5fd-ed7a0840d8fc admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:51.676364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 914/1823] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:51.710343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c172193-e183-4ca8-a01f-428567c3a8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:51.712111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c172193-e183-4ca8-a01f-428567c3a8ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:51.712812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c172193-e183-4ca8-a01f-428567c3a8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:51.713433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c172193-e183-4ca8-a01f-428567c3a8ec admin admin] PATH_INFO: `/v3/projects/d3f13655722d4a43803ac347df970a7a/users/bbca1ba75c894f209d733add6e78e24f/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:51.751408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d44b7a-5601-43d8-9872-956cebcec888 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:51.753102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b4d44b7a-5601-43d8-9872-956cebcec888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:51.761863 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c172193-e183-4ca8-a01f-428567c3a8ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f13655722d4a43803ac347df970a7a, user_id=bbca1ba75c894f209d733add6e78e24f, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:51.774852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c172193-e183-4ca8-a01f-428567c3a8ec admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:51.780117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d44b7a-5601-43d8-9872-956cebcec888 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:51.797582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:51.802799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:51.803681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:51.804467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] PATH_INFO: `/v3/users/b6bf87cb99e04cdfab0d0c9552aecf04` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:51.809513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6bf87cb99e04cdfab0d0c9552aecf04)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:51.819681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:51.839206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 910/1824] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:51 2022] PUT /identity/v3/projects/d3f13655722d4a43803ac347df970a7a/users/bbca1ba75c894f209d733add6e78e24f/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:51.844185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c172193-e183-4ca8-a01f-428567c3a8ec admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:51.964015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6bf87cb99e04cdfab0d0c9552aecf04'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:51.983087 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6bf87cb99e04cdfab0d0c9552aecf04'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:51.984637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6bf87cb99e04cdfab0d0c9552aecf04'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:52.007291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6bf87cb99e04cdfab0d0c9552aecf04'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:52.008445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6bf87cb99e04cdfab0d0c9552aecf04'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:52.021581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:52.057243 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:52.071408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c172193-e183-4ca8-a01f-428567c3a8ec admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:52.088417 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7490bd-6fac-4fea-be53-f7eed9a2fdb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:52.090943 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7490bd-6fac-4fea-be53-f7eed9a2fdb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:52.091141 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 915/1825] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:51 2022] DELETE /identity/v3/users/b6bf87cb99e04cdfab0d0c9552aecf04 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:52.092759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7490bd-6fac-4fea-be53-f7eed9a2fdb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:52.093489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7490bd-6fac-4fea-be53-f7eed9a2fdb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:52.096266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:52.099081 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:52.100320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bc7490bd-6fac-4fea-be53-f7eed9a2fdb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:52.454833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538dd042-78b2-4838-a6d6-454569661bf3 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:52.475058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64316d07-46f9-40d8-9717-031883c4fc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:52.475941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64316d07-46f9-40d8-9717-031883c4fc73 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:52.476266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64316d07-46f9-40d8-9717-031883c4fc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:52.476567 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64316d07-46f9-40d8-9717-031883c4fc73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:52.477327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64316d07-46f9-40d8-9717-031883c4fc73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:52.479111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64316d07-46f9-40d8-9717-031883c4fc73 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:52.502383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 916/1826] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:52.504537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7490bd-6fac-4fea-be53-f7eed9a2fdb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:52.508191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7490bd-6fac-4fea-be53-f7eed9a2fdb0 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:52.509031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b181-90f2-453a-b927-01e9e5021cf7 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:52.510515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b181-90f2-453a-b927-01e9e5021cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:52.510515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b181-90f2-453a-b927-01e9e5021cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:52.543453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 911/1827] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:52.544030 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7490bd-6fac-4fea-be53-f7eed9a2fdb0 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:52.554347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4d20b181-90f2-453a-b927-01e9e5021cf7 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:52.555966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4d20b181-90f2-453a-b927-01e9e5021cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:52.571702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7490bd-6fac-4fea-be53-f7eed9a2fdb0 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:52.584848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 917/1828] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:52.600754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96087fe2-9c3f-4372-b604-57b539d312dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:52.603363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96087fe2-9c3f-4372-b604-57b539d312dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:52.612279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96087fe2-9c3f-4372-b604-57b539d312dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:52.613568 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96087fe2-9c3f-4372-b604-57b539d312dc admin admin] PATH_INFO: `/v3/projects/d3f13655722d4a43803ac347df970a7a/users/bbca1ba75c894f209d733add6e78e24f/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:52.629562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96087fe2-9c3f-4372-b604-57b539d312dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f13655722d4a43803ac347df970a7a, user_id=bbca1ba75c894f209d733add6e78e24f, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:52.650995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96087fe2-9c3f-4372-b604-57b539d312dc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:52.705981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 912/1829] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:52 2022] PUT /identity/v3/projects/d3f13655722d4a43803ac347df970a7a/users/bbca1ba75c894f209d733add6e78e24f/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:52.717725 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a9549-f644-492f-92c3-f262f21142db None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:52.717992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a9549-f644-492f-92c3-f262f21142db None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:52.718294 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a9549-f644-492f-92c3-f262f21142db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:52.767872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d99a9549-f644-492f-92c3-f262f21142db None None] MFA Rules not processed for user `bbca1ba75c894f209d733add6e78e24f`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:52.769548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d99a9549-f644-492f-92c3-f262f21142db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:52.852635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96087fe2-9c3f-4372-b604-57b539d312dc admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:52.863235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-96087fe2-9c3f-4372-b604-57b539d312dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:52.927930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 918/1830] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:52.929439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a9549-f644-492f-92c3-f262f21142db None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:52.932153 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d99a9549-f644-492f-92c3-f262f21142db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:53.167976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a9549-f644-492f-92c3-f262f21142db None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:53.184865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ad8ec-9cc2-4e9c-9fba-035bb00b28f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:53.185700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ad8ec-9cc2-4e9c-9fba-035bb00b28f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:53.185991 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ad8ec-9cc2-4e9c-9fba-035bb00b28f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:53.205022 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ad8ec-9cc2-4e9c-9fba-035bb00b28f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:53.250362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ad8ec-9cc2-4e9c-9fba-035bb00b28f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:53.252262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ad8ec-9cc2-4e9c-9fba-035bb00b28f7 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:53.283906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 919/1831] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:53.386142 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96087fe2-9c3f-4372-b604-57b539d312dc admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:53.411944 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf46149-3576-487c-bc6e-3aedc03bd1f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:53.414320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf46149-3576-487c-bc6e-3aedc03bd1f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:53.414716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf46149-3576-487c-bc6e-3aedc03bd1f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:53.415269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf46149-3576-487c-bc6e-3aedc03bd1f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:53.770680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf46149-3576-487c-bc6e-3aedc03bd1f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:53.772507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf46149-3576-487c-bc6e-3aedc03bd1f1 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:53.862246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 913/1832] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 1014 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:54.584496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ad8ec-9cc2-4e9c-9fba-035bb00b28f7 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:54.602406 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ad8ec-9cc2-4e9c-9fba-035bb00b28f7 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:54.626553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:54.627495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:54.627790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:54.628076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] PATH_INFO: `/v3/users/1623525da690405c9c4bcf5e66eb122d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:54.636683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1623525da690405c9c4bcf5e66eb122d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:54.638700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:54.699009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:54.867590 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1623525da690405c9c4bcf5e66eb122d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:54.887258 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1623525da690405c9c4bcf5e66eb122d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:54.901235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1623525da690405c9c4bcf5e66eb122d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:54.940223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1623525da690405c9c4bcf5e66eb122d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:54.944119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1623525da690405c9c4bcf5e66eb122d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:55.057160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 920/1833] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:54 2022] DELETE /identity/v3/users/1623525da690405c9c4bcf5e66eb122d => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:55.063970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5c7be-28a1-477a-9a80-26360e4686f5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:55.064972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5c7be-28a1-477a-9a80-26360e4686f5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:55.065138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5c7be-28a1-477a-9a80-26360e4686f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:55.133234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-87f5c7be-28a1-477a-9a80-26360e4686f5 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:55.137775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-87f5c7be-28a1-477a-9a80-26360e4686f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:55.223520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:55.309962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 914/1834] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:55.478936 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:55.483654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:55.708729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab54042-9030-469a-8d7c-92a132f38d8a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:55.728271 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702bc15-d2dc-49f5-a7e8-c8931f8da099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:55.728976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702bc15-d2dc-49f5-a7e8-c8931f8da099 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:55.729780 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702bc15-d2dc-49f5-a7e8-c8931f8da099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:55.730059 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702bc15-d2dc-49f5-a7e8-c8931f8da099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:55.756352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702bc15-d2dc-49f5-a7e8-c8931f8da099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:55.757961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702bc15-d2dc-49f5-a7e8-c8931f8da099 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:55.801927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 921/1835] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:07:57.576523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5065d-ca6a-44c8-8a92-d2f7a1b8f5e2 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:57.577032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5065d-ca6a-44c8-8a92-d2f7a1b8f5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:57.577032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5065d-ca6a-44c8-8a92-d2f7a1b8f5e2 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:57.578374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 915/1836] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 11:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:07:57.643710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc613f3-3786-4858-a249-73a14bea6f39 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:57.648879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc613f3-3786-4858-a249-73a14bea6f39 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:57.649937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc613f3-3786-4858-a249-73a14bea6f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:57.800274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ddc613f3-3786-4858-a249-73a14bea6f39 None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:57.801572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ddc613f3-3786-4858-a249-73a14bea6f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:58.024758 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 922/1837] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 11:07:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:58.308137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5065d-ca6a-44c8-8a92-d2f7a1b8f5e2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:58.309976 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9aa5065d-ca6a-44c8-8a92-d2f7a1b8f5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:58.341920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5065d-ca6a-44c8-8a92-d2f7a1b8f5e2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:58.353668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:58.355289 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:58.355603 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:58.355886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] PATH_INFO: `/v3/projects/f74bd4caec0949b59ede43acfb0bcc1f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:58.361090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f74bd4caec0949b59ede43acfb0bcc1f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:58.363250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:58.492514 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f74bd4caec0949b59ede43acfb0bcc1f', 'request_id': 'req-5d127c05-9645-4de3-8dda-c63311f74e95'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:58.522046 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f74bd4caec0949b59ede43acfb0bcc1f', 'request_id': 'req-5d127c05-9645-4de3-8dda-c63311f74e95'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:58.546522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f74bd4caec0949b59ede43acfb0bcc1f', 'request_id': 'req-5d127c05-9645-4de3-8dda-c63311f74e95'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:58.576600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] The token cache is being invalidate because project f74bd4caec0949b59ede43acfb0bcc1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:58.577679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:58.578557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:58.583368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 916/1838] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:58 2022] DELETE /identity/v3/projects/f74bd4caec0949b59ede43acfb0bcc1f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:07:58.584741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc613f3-3786-4858-a249-73a14bea6f39 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:58.624709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc613f3-3786-4858-a249-73a14bea6f39 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:58.636776 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:58.638951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:58.682686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:58.798434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:58.799395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:58.800842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:58.801129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] PATH_INFO: `/v3/users/30e75169ab934d3495260d44618d134f` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:58.803331 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30e75169ab934d3495260d44618d134f)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:58.806538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:58.833678 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d127c05-9645-4de3-8dda-c63311f74e95 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:58.854720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c1a3a-63c9-41f2-9db7-cee3039a3130 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:58.857160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c1a3a-63c9-41f2-9db7-cee3039a3130 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:58.857566 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c1a3a-63c9-41f2-9db7-cee3039a3130 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:58.857947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c1a3a-63c9-41f2-9db7-cee3039a3130 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:58.870795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6a5c1a3a-63c9-41f2-9db7-cee3039a3130 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:58.902154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5c1a3a-63c9-41f2-9db7-cee3039a3130 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:58.904138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5c1a3a-63c9-41f2-9db7-cee3039a3130 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:58.934963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 917/1839] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:07:58.938708 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c1a3a-63c9-41f2-9db7-cee3039a3130 service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:58.943419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6a5c1a3a-63c9-41f2-9db7-cee3039a3130 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:58.970556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30e75169ab934d3495260d44618d134f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:58.981724 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c1a3a-63c9-41f2-9db7-cee3039a3130 service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:58.997561 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:58.998441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:58.998725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:58.999227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] PATH_INFO: `/v3/projects/0d4bd52b13e04ef9a7143d5b3f352339` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:59.000295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30e75169ab934d3495260d44618d134f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.001105 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d4bd52b13e04ef9a7143d5b3f352339)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:59.001423 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30e75169ab934d3495260d44618d134f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.003955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:59.033552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30e75169ab934d3495260d44618d134f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.035351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30e75169ab934d3495260d44618d134f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.064262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:59.105666 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 923/1840] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:58 2022] DELETE /identity/v3/users/30e75169ab934d3495260d44618d134f => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:59.108951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d4bd52b13e04ef9a7143d5b3f352339', 'request_id': 'req-fc07be66-dd90-4ccf-b556-92221556b964'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.114751 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:59.119540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:59.125494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d4bd52b13e04ef9a7143d5b3f352339', 'request_id': 'req-fc07be66-dd90-4ccf-b556-92221556b964'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.138718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d4bd52b13e04ef9a7143d5b3f352339', 'request_id': 'req-fc07be66-dd90-4ccf-b556-92221556b964'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.175360 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] The token cache is being invalidate because project 0d4bd52b13e04ef9a7143d5b3f352339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:59.176001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.180490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.183940 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 918/1841] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:58 2022] DELETE /identity/v3/projects/0d4bd52b13e04ef9a7143d5b3f352339 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:59.199833 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:59.208039 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:59.343660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42032a31-dcd3-41a0-a7ab-9cec3afeb796 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:59.362308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:59.363755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:59.363755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:59.363894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] PATH_INFO: `/v3/projects/ed7e3f280cb643d3be3730f787ffadff` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:59.379755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed7e3f280cb643d3be3730f787ffadff)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:59.388230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:59.468056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07be66-dd90-4ccf-b556-92221556b964 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:59.496361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df735773-4682-41d9-bec5-0a594689af42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:59.497654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df735773-4682-41d9-bec5-0a594689af42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:59.498073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df735773-4682-41d9-bec5-0a594689af42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:59.498591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df735773-4682-41d9-bec5-0a594689af42 admin admin] PATH_INFO: `/v3/users/bbca1ba75c894f209d733add6e78e24f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:59.506430 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df735773-4682-41d9-bec5-0a594689af42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbca1ba75c894f209d733add6e78e24f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:59.509173 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df735773-4682-41d9-bec5-0a594689af42 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:59.575552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:59.630704 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed7e3f280cb643d3be3730f787ffadff', 'request_id': 'req-1a1853b3-4999-40ba-83b1-664f1b5c3fde'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.639751 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-df735773-4682-41d9-bec5-0a594689af42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbca1ba75c894f209d733add6e78e24f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.640494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-df735773-4682-41d9-bec5-0a594689af42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbca1ba75c894f209d733add6e78e24f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.642419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed7e3f280cb643d3be3730f787ffadff', 'request_id': 'req-1a1853b3-4999-40ba-83b1-664f1b5c3fde'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.652701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed7e3f280cb643d3be3730f787ffadff', 'request_id': 'req-1a1853b3-4999-40ba-83b1-664f1b5c3fde'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.655271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-df735773-4682-41d9-bec5-0a594689af42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbca1ba75c894f209d733add6e78e24f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.670645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-df735773-4682-41d9-bec5-0a594689af42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbca1ba75c894f209d733add6e78e24f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.671424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-df735773-4682-41d9-bec5-0a594689af42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbca1ba75c894f209d733add6e78e24f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.676733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] The token cache is being invalidate because project ed7e3f280cb643d3be3730f787ffadff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:59.677528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.678655 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:59.686049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 924/1842] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:59 2022] DELETE /identity/v3/projects/ed7e3f280cb643d3be3730f787ffadff => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:59.688430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:59.691495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:59.718251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 919/1843] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:59 2022] DELETE /identity/v3/users/bbca1ba75c894f209d733add6e78e24f => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:59.719626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d10e2-8af9-4c13-9fd4-c438de1c7d18 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:59.719920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d10e2-8af9-4c13-9fd4-c438de1c7d18 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:59.720202 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d10e2-8af9-4c13-9fd4-c438de1c7d18 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:59.721811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 920/1844] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 11:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:07:59.789201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fecdb-a732-4f87-bf06-68d941ab9c8e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:59.789782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fecdb-a732-4f87-bf06-68d941ab9c8e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:59.790289 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fecdb-a732-4f87-bf06-68d941ab9c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:59.825164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-320fecdb-a732-4f87-bf06-68d941ab9c8e None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:59.826522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-320fecdb-a732-4f87-bf06-68d941ab9c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:59.840883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1853b3-4999-40ba-83b1-664f1b5c3fde admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:00.014902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:00.015530 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:00.016696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:00.016918 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:00.017202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] PATH_INFO: `/v3/users/2567d96a51234ce8892251dab738e9f7` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:00.020101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2567d96a51234ce8892251dab738e9f7)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:00.022171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:00.107783 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 921/1845] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 11:07:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:00.113824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2567d96a51234ce8892251dab738e9f7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:00.131173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2567d96a51234ce8892251dab738e9f7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:00.132203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2567d96a51234ce8892251dab738e9f7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:00.144363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2567d96a51234ce8892251dab738e9f7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:00.145500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2567d96a51234ce8892251dab738e9f7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:00.163105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:00.177481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 925/1846] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:59 2022] DELETE /identity/v3/users/2567d96a51234ce8892251dab738e9f7 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:00.368707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fecdb-a732-4f87-bf06-68d941ab9c8e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:00.371981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-320fecdb-a732-4f87-bf06-68d941ab9c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:00.390746 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fecdb-a732-4f87-bf06-68d941ab9c8e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:00.484084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c755162-f565-4ad5-bf58-f7dd9d52ac5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:00.484781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c755162-f565-4ad5-bf58-f7dd9d52ac5a service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:00.485068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c755162-f565-4ad5-bf58-f7dd9d52ac5a service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:00.485577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c755162-f565-4ad5-bf58-f7dd9d52ac5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:00.488805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8c755162-f565-4ad5-bf58-f7dd9d52ac5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:00.504745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c755162-f565-4ad5-bf58-f7dd9d52ac5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:00.507007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c755162-f565-4ad5-bf58-f7dd9d52ac5a service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:00.523941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 922/1847] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:00.665415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:00.673198 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:00.746076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c755162-f565-4ad5-bf58-f7dd9d52ac5a service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:00.748272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8c755162-f565-4ad5-bf58-f7dd9d52ac5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:00.800786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750577fd-7d5e-41be-aaca-fb9835e96abe admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:00.807004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:00.808514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:00.808879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:00.809627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] PATH_INFO: `/v3/projects/d3f13655722d4a43803ac347df970a7a` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:00.811581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f13655722d4a43803ac347df970a7a)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:00.816548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:00.875060 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c755162-f565-4ad5-bf58-f7dd9d52ac5a service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:00.890705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:00.892243 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:00.894414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:00.894772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22 admin admin] PATH_INFO: `/v3/projects/e3462f63802848e49d2be9d05b39cc70` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:00.897197 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3462f63802848e49d2be9d05b39cc70)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:00.900693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:00.913740 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f13655722d4a43803ac347df970a7a', 'request_id': 'req-3a68b0fc-4831-4663-a463-be9686b9b19c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:00.921702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f13655722d4a43803ac347df970a7a', 'request_id': 'req-3a68b0fc-4831-4663-a463-be9686b9b19c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:00.928002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f13655722d4a43803ac347df970a7a', 'request_id': 'req-3a68b0fc-4831-4663-a463-be9686b9b19c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:00.952908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] The token cache is being invalidate because project d3f13655722d4a43803ac347df970a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:00.953483 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:00.954403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:00.958106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 926/1848] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:00 2022] DELETE /identity/v3/projects/d3f13655722d4a43803ac347df970a7a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:00.959683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:00.961899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:01.023920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3462f63802848e49d2be9d05b39cc70', 'request_id': 'req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:01.044000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3462f63802848e49d2be9d05b39cc70', 'request_id': 'req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:01.049994 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:01.057947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3462f63802848e49d2be9d05b39cc70', 'request_id': 'req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:01.090180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22 admin admin] The token cache is being invalidate because project e3462f63802848e49d2be9d05b39cc70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:01.090180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:01.092912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-227ab1ac-a6e8-4dc7-a12f-ce4115266c22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:01.096131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 923/1849] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:00 2022] DELETE /identity/v3/projects/e3462f63802848e49d2be9d05b39cc70 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:01.102892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc27f51-6099-43a4-b7b6-fbd3561fd74e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.103186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc27f51-6099-43a4-b7b6-fbd3561fd74e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.107223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc27f51-6099-43a4-b7b6-fbd3561fd74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.120358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68b0fc-4831-4663-a463-be9686b9b19c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.137486 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2fc27f51-6099-43a4-b7b6-fbd3561fd74e None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:01.138767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2fc27f51-6099-43a4-b7b6-fbd3561fd74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:01.192793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:01.247183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 924/1850] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:01.248652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f13119-0128-4cf1-95dd-fe415ad6390b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.248946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f13119-0128-4cf1-95dd-fe415ad6390b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.249231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f13119-0128-4cf1-95dd-fe415ad6390b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.255317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.258091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.259070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.261254 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36 admin admin] PATH_INFO: `/v3/projects/261d958c5a8847178473cab5b663d024` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.263787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=261d958c5a8847178473cab5b663d024)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.266877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.271194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-86f13119-0128-4cf1-95dd-fe415ad6390b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:01.272367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-86f13119-0128-4cf1-95dd-fe415ad6390b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:01.284071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 925/1851] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:01.287124 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a67e43-bc4e-4220-b85d-900c64da5860 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.287632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a67e43-bc4e-4220-b85d-900c64da5860 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.288121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a67e43-bc4e-4220-b85d-900c64da5860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.319123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e8a67e43-bc4e-4220-b85d-900c64da5860 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:01.320338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e8a67e43-bc4e-4220-b85d-900c64da5860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:01.345046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '261d958c5a8847178473cab5b663d024', 'request_id': 'req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:01.351691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '261d958c5a8847178473cab5b663d024', 'request_id': 'req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:01.357481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '261d958c5a8847178473cab5b663d024', 'request_id': 'req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:01.371848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36 admin admin] The token cache is being invalidate because project 261d958c5a8847178473cab5b663d024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:01.372136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:01.372866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4c5fe39a-1dbe-459c-8258-7ebe9ded4d36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:01.374514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 927/1852] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:00 2022] DELETE /identity/v3/projects/261d958c5a8847178473cab5b663d024 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:01.375823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c2e21-052b-4237-b420-849f651ded1d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.376109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c2e21-052b-4237-b420-849f651ded1d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.376380 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c2e21-052b-4237-b420-849f651ded1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.400254 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-856c2e21-052b-4237-b420-849f651ded1d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:01.401899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-856c2e21-052b-4237-b420-849f651ded1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:01.402088 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 926/1853] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:01.403786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8340e-4b6d-44dc-9808-572d69caf414 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.404204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8340e-4b6d-44dc-9808-572d69caf414 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.404490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8340e-4b6d-44dc-9808-572d69caf414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.411247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 928/1854] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:01.416291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c2e21-052b-4237-b420-849f651ded1d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.425109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-eba8340e-4b6d-44dc-9808-572d69caf414 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:01.426347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-eba8340e-4b6d-44dc-9808-572d69caf414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:01.428350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c2e21-052b-4237-b420-849f651ded1d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.434877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62024384-5e6d-48bd-ba24-d69f785f94a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.435606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62024384-5e6d-48bd-ba24-d69f785f94a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.435721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 927/1855] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:01.436036 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62024384-5e6d-48bd-ba24-d69f785f94a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.436315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62024384-5e6d-48bd-ba24-d69f785f94a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.436589 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8340e-4b6d-44dc-9808-572d69caf414 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.437116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62024384-5e6d-48bd-ba24-d69f785f94a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:01.437428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62024384-5e6d-48bd-ba24-d69f785f94a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.439072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62024384-5e6d-48bd-ba24-d69f785f94a5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.448639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8340e-4b6d-44dc-9808-572d69caf414 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.453878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc979ad-1170-4e21-91bf-2c0775653721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.454564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc979ad-1170-4e21-91bf-2c0775653721 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.454849 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc979ad-1170-4e21-91bf-2c0775653721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.455174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc979ad-1170-4e21-91bf-2c0775653721 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.456182 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc979ad-1170-4e21-91bf-2c0775653721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:01.456382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 929/1856] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:08:01.457802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487c281-9602-42eb-9e20-862c0fd8a70b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.458102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487c281-9602-42eb-9e20-862c0fd8a70b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.458602 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487c281-9602-42eb-9e20-862c0fd8a70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.460820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc979ad-1170-4e21-91bf-2c0775653721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.462646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc979ad-1170-4e21-91bf-2c0775653721 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.478844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 928/1857] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:08:01.479886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc979ad-1170-4e21-91bf-2c0775653721 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.487170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2487c281-9602-42eb-9e20-862c0fd8a70b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:01.488731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2487c281-9602-42eb-9e20-862c0fd8a70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:01.498202 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc979ad-1170-4e21-91bf-2c0775653721 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.505587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 930/1858] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:01.506635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487c281-9602-42eb-9e20-862c0fd8a70b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.507409 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd560999-c9c7-4ea2-9e75-579c0dac34e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.508440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd560999-c9c7-4ea2-9e75-579c0dac34e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.509676 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd560999-c9c7-4ea2-9e75-579c0dac34e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.509994 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd560999-c9c7-4ea2-9e75-579c0dac34e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.511715 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd560999-c9c7-4ea2-9e75-579c0dac34e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.513838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd560999-c9c7-4ea2-9e75-579c0dac34e0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.518650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487c281-9602-42eb-9e20-862c0fd8a70b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.523880 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009ed80-fb3f-43da-9950-113e1e92e668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.524565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009ed80-fb3f-43da-9950-113e1e92e668 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.524940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009ed80-fb3f-43da-9950-113e1e92e668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.525121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009ed80-fb3f-43da-9950-113e1e92e668 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.526424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009ed80-fb3f-43da-9950-113e1e92e668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.528416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009ed80-fb3f-43da-9950-113e1e92e668 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.565121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 929/1859] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:01 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:01.566302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd560999-c9c7-4ea2-9e75-579c0dac34e0 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.576589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 931/1860] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:01 2022] POST /identity/v3/projects => generated 401 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:01.581126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009ed80-fb3f-43da-9950-113e1e92e668 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.646843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:01.650177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009ed80-fb3f-43da-9950-113e1e92e668 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.656276 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbcab0-ca35-4eab-981d-ecb62d7e184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.657219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbcab0-ca35-4eab-981d-ecb62d7e184d admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.657398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbcab0-ca35-4eab-981d-ecb62d7e184d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.657676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbcab0-ca35-4eab-981d-ecb62d7e184d admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.658365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbcab0-ca35-4eab-981d-ecb62d7e184d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.660345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbcab0-ca35-4eab-981d-ecb62d7e184d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.669343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd560999-c9c7-4ea2-9e75-579c0dac34e0 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.674397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185243c-596e-4d0c-b4a7-6e3d33768e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.675076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185243c-596e-4d0c-b4a7-6e3d33768e69 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.675531 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185243c-596e-4d0c-b4a7-6e3d33768e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.675821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185243c-596e-4d0c-b4a7-6e3d33768e69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.676559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185243c-596e-4d0c-b4a7-6e3d33768e69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:01.676869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185243c-596e-4d0c-b4a7-6e3d33768e69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.678491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185243c-596e-4d0c-b4a7-6e3d33768e69 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.684990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 932/1861] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:01 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:01.684990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbcab0-ca35-4eab-981d-ecb62d7e184d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.688514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:01.694938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 930/1862] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:08:01.695125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbcab0-ca35-4eab-981d-ecb62d7e184d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.696421 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185243c-596e-4d0c-b4a7-6e3d33768e69 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.701808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176520d5-c689-43db-8e4e-debd2568abd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.702691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176520d5-c689-43db-8e4e-debd2568abd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.703206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176520d5-c689-43db-8e4e-debd2568abd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.703755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176520d5-c689-43db-8e4e-debd2568abd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.704559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176520d5-c689-43db-8e4e-debd2568abd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.706759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176520d5-c689-43db-8e4e-debd2568abd2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.719555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185243c-596e-4d0c-b4a7-6e3d33768e69 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.725255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54167-caa2-4346-852d-073244628c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.725930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54167-caa2-4346-852d-073244628c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.726217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54167-caa2-4346-852d-073244628c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.726491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54167-caa2-4346-852d-073244628c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.727218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf54167-caa2-4346-852d-073244628c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.731489 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 933/1863] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:01 2022] POST /identity/v3/users => generated 301 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:01.731489 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176520d5-c689-43db-8e4e-debd2568abd2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.731746 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf54167-caa2-4346-852d-073244628c20 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.739860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176520d5-c689-43db-8e4e-debd2568abd2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.741972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 931/1864] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:01.742882 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54167-caa2-4346-852d-073244628c20 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.747271 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ab2a1-2be5-4f36-b7a3-6f6887fcf85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.748120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ab2a1-2be5-4f36-b7a3-6f6887fcf85d admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.750346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ab2a1-2be5-4f36-b7a3-6f6887fcf85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.750651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ab2a1-2be5-4f36-b7a3-6f6887fcf85d admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.751529 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ab2a1-2be5-4f36-b7a3-6f6887fcf85d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.753507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ab2a1-2be5-4f36-b7a3-6f6887fcf85d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.755323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54167-caa2-4346-852d-073244628c20 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.762716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1c447-3825-45c6-bb7f-2d62d2255635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.763702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1c447-3825-45c6-bb7f-2d62d2255635 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.764267 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1c447-3825-45c6-bb7f-2d62d2255635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.764808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1c447-3825-45c6-bb7f-2d62d2255635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.765769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1c447-3825-45c6-bb7f-2d62d2255635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.767831 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1c447-3825-45c6-bb7f-2d62d2255635 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.779156 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 932/1865] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:01.780346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1c447-3825-45c6-bb7f-2d62d2255635 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.784167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 934/1866] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:01 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:01.785099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ab2a1-2be5-4f36-b7a3-6f6887fcf85d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.796454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1c447-3825-45c6-bb7f-2d62d2255635 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.882258 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ab2a1-2be5-4f36-b7a3-6f6887fcf85d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.886449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28842c-90e6-47c5-865e-65ad3fe756ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.886449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28842c-90e6-47c5-865e-65ad3fe756ec admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.886732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28842c-90e6-47c5-865e-65ad3fe756ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.886935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28842c-90e6-47c5-865e-65ad3fe756ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.887890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28842c-90e6-47c5-865e-65ad3fe756ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.888827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424456c-5714-4eb7-87fd-948bdb21b34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.889594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424456c-5714-4eb7-87fd-948bdb21b34e admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.889711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28842c-90e6-47c5-865e-65ad3fe756ec admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.889946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424456c-5714-4eb7-87fd-948bdb21b34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.890259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424456c-5714-4eb7-87fd-948bdb21b34e admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.891213 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424456c-5714-4eb7-87fd-948bdb21b34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.892956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424456c-5714-4eb7-87fd-948bdb21b34e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.906862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 933/1867] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:01.906862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28842c-90e6-47c5-865e-65ad3fe756ec admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.907136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 935/1868] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:01.908051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424456c-5714-4eb7-87fd-948bdb21b34e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.922725 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424456c-5714-4eb7-87fd-948bdb21b34e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.923538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28842c-90e6-47c5-865e-65ad3fe756ec admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:01.928321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c23e7-8825-4644-bee9-a8cd62f094f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.928625 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aab322-3ede-4653-bc0e-265723d17df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:01.929361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c23e7-8825-4644-bee9-a8cd62f094f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.929977 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c23e7-8825-4644-bee9-a8cd62f094f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.930171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aab322-3ede-4653-bc0e-265723d17df7 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:01.930321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c23e7-8825-4644-bee9-a8cd62f094f4 admin admin] PATH_INFO: `/v3/projects/758d2f7671144774a12542f664f2f028/users/1262f95bcaa24ff69e65ecb88a21f711/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.930463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aab322-3ede-4653-bc0e-265723d17df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:01.930739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aab322-3ede-4653-bc0e-265723d17df7 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:01.931958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aab322-3ede-4653-bc0e-265723d17df7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.936993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aab322-3ede-4653-bc0e-265723d17df7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.952569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c23e7-8825-4644-bee9-a8cd62f094f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=758d2f7671144774a12542f664f2f028, user_id=1262f95bcaa24ff69e65ecb88a21f711, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:01.956571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c23e7-8825-4644-bee9-a8cd62f094f4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:01.972497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 934/1869] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:01 2022] POST /identity/v3/users => generated 289 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:01.973733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aab322-3ede-4653-bc0e-265723d17df7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.976029 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 936/1870] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:01 2022] PUT /identity/v3/projects/758d2f7671144774a12542f664f2f028/users/1262f95bcaa24ff69e65ecb88a21f711/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:01.977076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c23e7-8825-4644-bee9-a8cd62f094f4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:01.986390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aab322-3ede-4653-bc0e-265723d17df7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:02.071253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c23e7-8825-4644-bee9-a8cd62f094f4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:02.080552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed4693-fc19-4a26-ab17-19eeef679caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:02.081614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed4693-fc19-4a26-ab17-19eeef679caf admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:02.082775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed4693-fc19-4a26-ab17-19eeef679caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:02.082915 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b9e36-3fb6-410e-bf2b-84dd72803e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:02.083012 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed4693-fc19-4a26-ab17-19eeef679caf admin admin] PATH_INFO: `/v3/projects/ffef335cbe6846248ed7d672d7acd424/users/c609de2bb6ca498ca2c0250b8c3e4f56/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:02.087289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b9e36-3fb6-410e-bf2b-84dd72803e1f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:02.087651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b9e36-3fb6-410e-bf2b-84dd72803e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:02.088095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b9e36-3fb6-410e-bf2b-84dd72803e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:02.089506 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b9e36-3fb6-410e-bf2b-84dd72803e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:02.091430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b9e36-3fb6-410e-bf2b-84dd72803e1f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:02.106001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed4693-fc19-4a26-ab17-19eeef679caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffef335cbe6846248ed7d672d7acd424, user_id=c609de2bb6ca498ca2c0250b8c3e4f56, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:02.110164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed4693-fc19-4a26-ab17-19eeef679caf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:02.118697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 937/1871] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:02.121057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935d10e-8da8-4ae6-b21d-c5017df42a6c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:02.121359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935d10e-8da8-4ae6-b21d-c5017df42a6c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:02.121642 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935d10e-8da8-4ae6-b21d-c5017df42a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:02.143087 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 935/1872] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:01 2022] PUT /identity/v3/projects/ffef335cbe6846248ed7d672d7acd424/users/c609de2bb6ca498ca2c0250b8c3e4f56/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:02.144405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed4693-fc19-4a26-ab17-19eeef679caf admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:02.161243 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:02.165856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5935d10e-8da8-4ae6-b21d-c5017df42a6c None None] MFA Rules not processed for user `1262f95bcaa24ff69e65ecb88a21f711`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:02.168295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5935d10e-8da8-4ae6-b21d-c5017df42a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:02.200677 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:02.259321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed4693-fc19-4a26-ab17-19eeef679caf admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:02.266059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a367-a9d5-44ec-ab9d-07408676152f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:02.267315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a367-a9d5-44ec-ab9d-07408676152f admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:02.267942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a367-a9d5-44ec-ab9d-07408676152f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:02.268515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a367-a9d5-44ec-ab9d-07408676152f admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:02.269629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a367-a9d5-44ec-ab9d-07408676152f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:02.272387 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a367-a9d5-44ec-ab9d-07408676152f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:02.287287 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 936/1873] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:02.290224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff285d6-7f31-48f2-9463-df9c59ef29d8 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:02.290640 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff285d6-7f31-48f2-9463-df9c59ef29d8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:02.291038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff285d6-7f31-48f2-9463-df9c59ef29d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:02.308195 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 938/1874] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:02.308195 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935d10e-8da8-4ae6-b21d-c5017df42a6c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:02.331765 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935d10e-8da8-4ae6-b21d-c5017df42a6c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:02.335377 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3ff285d6-7f31-48f2-9463-df9c59ef29d8 None None] MFA Rules not processed for user `c609de2bb6ca498ca2c0250b8c3e4f56`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:02.338291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3ff285d6-7f31-48f2-9463-df9c59ef29d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:02.340818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc10d0-e698-4f7b-926c-2bad86d25792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:02.343646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc10d0-e698-4f7b-926c-2bad86d25792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:02.344171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc10d0-e698-4f7b-926c-2bad86d25792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:02.344674 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc10d0-e698-4f7b-926c-2bad86d25792 admin admin] PATH_INFO: `/v3/projects/3d1dbd18c98949fabf3a1dd1c1d9691a/users/d6b236f644b0405e8375b00e49d735ac/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:02.367117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc10d0-e698-4f7b-926c-2bad86d25792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1dbd18c98949fabf3a1dd1c1d9691a, user_id=d6b236f644b0405e8375b00e49d735ac, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:02.371374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc10d0-e698-4f7b-926c-2bad86d25792 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:02.409608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 939/1875] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:02 2022] PUT /identity/v3/projects/3d1dbd18c98949fabf3a1dd1c1d9691a/users/d6b236f644b0405e8375b00e49d735ac/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:02.413587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc10d0-e698-4f7b-926c-2bad86d25792 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:02.416082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3bfc10d0-e698-4f7b-926c-2bad86d25792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:02.491037 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 937/1876] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:02.492894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5b4a3-910a-4162-b061-ad0445ebc23b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:02.493306 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5b4a3-910a-4162-b061-ad0445ebc23b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:02.493697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5b4a3-910a-4162-b061-ad0445ebc23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:02.532063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-17e5b4a3-910a-4162-b061-ad0445ebc23b None None] MFA Rules not processed for user `d6b236f644b0405e8375b00e49d735ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:02.534367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-17e5b4a3-910a-4162-b061-ad0445ebc23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:02.568616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc10d0-e698-4f7b-926c-2bad86d25792 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:02.575244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8207f-76eb-4ff5-8360-4efd4d97c219 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1ad1ffe4d99420cb568d6154f779838', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:02.575983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8207f-76eb-4ff5-8360-4efd4d97c219 service glance] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:02.576276 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8207f-76eb-4ff5-8360-4efd4d97c219 service glance] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:02.576559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8207f-76eb-4ff5-8360-4efd4d97c219 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:02.690127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8207f-76eb-4ff5-8360-4efd4d97c219 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:02.693227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8207f-76eb-4ff5-8360-4efd4d97c219 service glance] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:02.707539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 938/1877] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:02.708691 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5b4a3-910a-4162-b061-ad0445ebc23b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:02.719993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 940/1878] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:02.720992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8207f-76eb-4ff5-8360-4efd4d97c219 service glance] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:02.756118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:02.799748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5b4a3-910a-4162-b061-ad0445ebc23b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:02.806167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21911c-213f-4662-9187-7822aefaf308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:02.807294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21911c-213f-4662-9187-7822aefaf308 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:02.807841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21911c-213f-4662-9187-7822aefaf308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:02.808348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21911c-213f-4662-9187-7822aefaf308 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:02.809288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21911c-213f-4662-9187-7822aefaf308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:02.812962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21911c-213f-4662-9187-7822aefaf308 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:02.826410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:02.872585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8207f-76eb-4ff5-8360-4efd4d97c219 service glance] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:02.873142 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 939/1879] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:02 2022] POST /identity/v3/projects => generated 401 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:02.874525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21911c-213f-4662-9187-7822aefaf308 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:02.948973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bba2d5-e459-4174-b905-9947fad4a732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:02.949743 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bba2d5-e459-4174-b905-9947fad4a732 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:02.950080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bba2d5-e459-4174-b905-9947fad4a732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:02.950469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bba2d5-e459-4174-b905-9947fad4a732 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:02.951158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bba2d5-e459-4174-b905-9947fad4a732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:02.953117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bba2d5-e459-4174-b905-9947fad4a732 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.000205 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 941/1880] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:02 2022] POST /identity/v3/projects => generated 379 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:03.001422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bba2d5-e459-4174-b905-9947fad4a732 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.021189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21911c-213f-4662-9187-7822aefaf308 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.090693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bba2d5-e459-4174-b905-9947fad4a732 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.102778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62edda0-9092-49dc-b0c6-113d2ed98e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.104698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62edda0-9092-49dc-b0c6-113d2ed98e0f admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.105552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62edda0-9092-49dc-b0c6-113d2ed98e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.106015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62edda0-9092-49dc-b0c6-113d2ed98e0f admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.107031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62edda0-9092-49dc-b0c6-113d2ed98e0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.109796 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62edda0-9092-49dc-b0c6-113d2ed98e0f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.116380 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40822406-e452-479e-bf5a-9ca0230e2ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.117223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40822406-e452-479e-bf5a-9ca0230e2ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.117510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40822406-e452-479e-bf5a-9ca0230e2ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.117820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40822406-e452-479e-bf5a-9ca0230e2ec9 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.118483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40822406-e452-479e-bf5a-9ca0230e2ec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.120662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40822406-e452-479e-bf5a-9ca0230e2ec9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.145664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 942/1881] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:03 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:03.148066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62edda0-9092-49dc-b0c6-113d2ed98e0f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.153054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 940/1882] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:02 2022] POST /identity/v3/users => generated 307 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:03.154017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40822406-e452-479e-bf5a-9ca0230e2ec9 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.162152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62edda0-9092-49dc-b0c6-113d2ed98e0f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.167812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40822406-e452-479e-bf5a-9ca0230e2ec9 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.169557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5e3b5-186b-4091-b20e-648cf993177a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.170965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5e3b5-186b-4091-b20e-648cf993177a admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.171496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5e3b5-186b-4091-b20e-648cf993177a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.171999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5e3b5-186b-4091-b20e-648cf993177a admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.173382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5e3b5-186b-4091-b20e-648cf993177a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.176828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5e3b5-186b-4091-b20e-648cf993177a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.181198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6a3ec-e2b6-42da-9f31-5fa2a970a372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.182440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6a3ec-e2b6-42da-9f31-5fa2a970a372 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.182771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6a3ec-e2b6-42da-9f31-5fa2a970a372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.183086 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6a3ec-e2b6-42da-9f31-5fa2a970a372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.184388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6a3ec-e2b6-42da-9f31-5fa2a970a372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.186546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6a3ec-e2b6-42da-9f31-5fa2a970a372 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.207169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 941/1883] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:08:03.212855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6a3ec-e2b6-42da-9f31-5fa2a970a372 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.213058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 943/1884] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:03.219714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5e3b5-186b-4091-b20e-648cf993177a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.225952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6a3ec-e2b6-42da-9f31-5fa2a970a372 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.233048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5038a-f3cf-4407-9d8f-0034e351b93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.234735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5038a-f3cf-4407-9d8f-0034e351b93c admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.235152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5038a-f3cf-4407-9d8f-0034e351b93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.235531 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5038a-f3cf-4407-9d8f-0034e351b93c admin admin] PATH_INFO: `/v3/projects/b7f363554d98430286c6b81cb6e5d7f7/users/07f63cff45d4483ca5c66f1f1da25ccc/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.240135 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5e3b5-186b-4091-b20e-648cf993177a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.251558 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b90b6c-7ca6-48aa-9e0a-6836d200a47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.252839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b90b6c-7ca6-48aa-9e0a-6836d200a47a admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.253121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b90b6c-7ca6-48aa-9e0a-6836d200a47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.253582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b90b6c-7ca6-48aa-9e0a-6836d200a47a admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.254849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b90b6c-7ca6-48aa-9e0a-6836d200a47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.257316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b90b6c-7ca6-48aa-9e0a-6836d200a47a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.259665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5038a-f3cf-4407-9d8f-0034e351b93c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f363554d98430286c6b81cb6e5d7f7, user_id=07f63cff45d4483ca5c66f1f1da25ccc, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.264789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5038a-f3cf-4407-9d8f-0034e351b93c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.280634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 944/1885] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:03.281919 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b90b6c-7ca6-48aa-9e0a-6836d200a47a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.286751 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 942/1886] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:03 2022] PUT /identity/v3/projects/b7f363554d98430286c6b81cb6e5d7f7/users/07f63cff45d4483ca5c66f1f1da25ccc/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:03.291020 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5038a-f3cf-4407-9d8f-0034e351b93c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.297049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b90b6c-7ca6-48aa-9e0a-6836d200a47a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.313347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:03.333895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:03.370607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0984e-5992-4539-8f8b-d59dfc1d294c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.371587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0984e-5992-4539-8f8b-d59dfc1d294c admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.371942 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0984e-5992-4539-8f8b-d59dfc1d294c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.372329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0984e-5992-4539-8f8b-d59dfc1d294c admin admin] PATH_INFO: `/v3/projects/5c6421bc17ca4436a75a4d95b9596d7f/users/0dbd6661f8694640b14dd9d647410b92/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.376597 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5038a-f3cf-4407-9d8f-0034e351b93c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.382957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c98842-18c5-49ae-be40-bac242e22c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.383866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c98842-18c5-49ae-be40-bac242e22c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.384138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c98842-18c5-49ae-be40-bac242e22c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.384531 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c98842-18c5-49ae-be40-bac242e22c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.385953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c98842-18c5-49ae-be40-bac242e22c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.388519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c98842-18c5-49ae-be40-bac242e22c4c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.393387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0984e-5992-4539-8f8b-d59dfc1d294c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6421bc17ca4436a75a4d95b9596d7f, user_id=0dbd6661f8694640b14dd9d647410b92, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.397800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0984e-5992-4539-8f8b-d59dfc1d294c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.403513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 943/1887] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:03.407210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c98842-18c5-49ae-be40-bac242e22c4c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.420551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 945/1888] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:03 2022] PUT /identity/v3/projects/5c6421bc17ca4436a75a4d95b9596d7f/users/0dbd6661f8694640b14dd9d647410b92/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:03.434451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf015e-d41b-454f-b8b3-4c00c5496b33 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.434451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf015e-d41b-454f-b8b3-4c00c5496b33 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.434451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf015e-d41b-454f-b8b3-4c00c5496b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.437507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c98842-18c5-49ae-be40-bac242e22c4c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.473464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-26bf015e-d41b-454f-b8b3-4c00c5496b33 None None] MFA Rules not processed for user `0dbd6661f8694640b14dd9d647410b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:03.475153 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-26bf015e-d41b-454f-b8b3-4c00c5496b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:03.549584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de07b7-f5b1-41f6-ac66-50c02f526896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.550506 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de07b7-f5b1-41f6-ac66-50c02f526896 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.551027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de07b7-f5b1-41f6-ac66-50c02f526896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.551315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de07b7-f5b1-41f6-ac66-50c02f526896 admin admin] PATH_INFO: `/v3/projects/b7f363554d98430286c6b81cb6e5d7f7/users/07f63cff45d4483ca5c66f1f1da25ccc/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.554339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de07b7-f5b1-41f6-ac66-50c02f526896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f363554d98430286c6b81cb6e5d7f7, user_id=07f63cff45d4483ca5c66f1f1da25ccc, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.558639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de07b7-f5b1-41f6-ac66-50c02f526896 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.569074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 946/1889] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:03.574027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf015e-d41b-454f-b8b3-4c00c5496b33 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.586608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf015e-d41b-454f-b8b3-4c00c5496b33 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.591646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 944/1890] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:03 2022] PUT /identity/v3/projects/b7f363554d98430286c6b81cb6e5d7f7/users/07f63cff45d4483ca5c66f1f1da25ccc/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:03.611715 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dcd60-b024-4607-9fc9-6ea4efbb73ce None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.612019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dcd60-b024-4607-9fc9-6ea4efbb73ce None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.612788 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dcd60-b024-4607-9fc9-6ea4efbb73ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.661120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1c1dcd60-b024-4607-9fc9-6ea4efbb73ce None None] MFA Rules not processed for user `07f63cff45d4483ca5c66f1f1da25ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:03.662854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1c1dcd60-b024-4607-9fc9-6ea4efbb73ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:03.697248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe2800-962e-4295-b1cb-0c46261dcca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.698748 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe2800-962e-4295-b1cb-0c46261dcca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.699034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe2800-962e-4295-b1cb-0c46261dcca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.699316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe2800-962e-4295-b1cb-0c46261dcca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.700335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe2800-962e-4295-b1cb-0c46261dcca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.702283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe2800-962e-4295-b1cb-0c46261dcca2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.743884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 947/1891] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:03 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:08:03.747369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe2800-962e-4295-b1cb-0c46261dcca2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.773275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 945/1892] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:03 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:03.784746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:03.786343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dcd60-b024-4607-9fc9-6ea4efbb73ce None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.787992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1c1dcd60-b024-4607-9fc9-6ea4efbb73ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:03.834768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:03.843573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe2800-962e-4295-b1cb-0c46261dcca2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.848522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5471b-6cbc-4d32-8a52-ed1c70c8daa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.849205 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5471b-6cbc-4d32-8a52-ed1c70c8daa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.849491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5471b-6cbc-4d32-8a52-ed1c70c8daa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.849928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5471b-6cbc-4d32-8a52-ed1c70c8daa6 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.850453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5471b-6cbc-4d32-8a52-ed1c70c8daa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.852465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5471b-6cbc-4d32-8a52-ed1c70c8daa6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.870737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dcd60-b024-4607-9fc9-6ea4efbb73ce None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.889903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 948/1893] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:03 2022] POST /identity/v3/users => generated 296 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:03.893739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5471b-6cbc-4d32-8a52-ed1c70c8daa6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.904783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5471b-6cbc-4d32-8a52-ed1c70c8daa6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.909948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ebffc-a830-48d8-a155-769d366651f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.910791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ebffc-a830-48d8-a155-769d366651f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.911249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ebffc-a830-48d8-a155-769d366651f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.911596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ebffc-a830-48d8-a155-769d366651f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.912539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ebffc-a830-48d8-a155-769d366651f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.914191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ebffc-a830-48d8-a155-769d366651f2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.928621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 949/1894] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:08:03.928816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f412e7-87ad-44ff-9270-a59e6bb25a42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.928816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f412e7-87ad-44ff-9270-a59e6bb25a42 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.928816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f412e7-87ad-44ff-9270-a59e6bb25a42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.928816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f412e7-87ad-44ff-9270-a59e6bb25a42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.929878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ebffc-a830-48d8-a155-769d366651f2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:03.940635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ebffc-a830-48d8-a155-769d366651f2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:03.945504 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748d099-afc8-4daa-aa84-cddb85186fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:03.946395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748d099-afc8-4daa-aa84-cddb85186fde admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:03.946736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748d099-afc8-4daa-aa84-cddb85186fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:03.947022 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748d099-afc8-4daa-aa84-cddb85186fde admin admin] PATH_INFO: `/v3/projects/f4eee425606941209f6ea7b27054a448/users/441bfaaf1b344b77be4f29567fe39fb4/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:03.971245 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e748d099-afc8-4daa-aa84-cddb85186fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4eee425606941209f6ea7b27054a448, user_id=441bfaaf1b344b77be4f29567fe39fb4, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:03.974088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e748d099-afc8-4daa-aa84-cddb85186fde admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:03.994620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 950/1895] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:03 2022] PUT /identity/v3/projects/f4eee425606941209f6ea7b27054a448/users/441bfaaf1b344b77be4f29567fe39fb4/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:03.999058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748d099-afc8-4daa-aa84-cddb85186fde admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:04.016936 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f412e7-87ad-44ff-9270-a59e6bb25a42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:04.018433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f412e7-87ad-44ff-9270-a59e6bb25a42 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:04.030307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 946/1896] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:03 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:08:04.078702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748d099-afc8-4daa-aa84-cddb85186fde admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:04.087104 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e646b86-2229-4e91-8358-0cf2c3090a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:04.087841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e646b86-2229-4e91-8358-0cf2c3090a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:04.088129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e646b86-2229-4e91-8358-0cf2c3090a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:04.088480 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e646b86-2229-4e91-8358-0cf2c3090a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:04.089187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e646b86-2229-4e91-8358-0cf2c3090a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:04.090831 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e646b86-2229-4e91-8358-0cf2c3090a4b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:04.101933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 951/1897] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:08:04.112485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f412e7-87ad-44ff-9270-a59e6bb25a42 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:04.123726 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e646b86-2229-4e91-8358-0cf2c3090a4b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:04.126987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f412e7-87ad-44ff-9270-a59e6bb25a42 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:04.132104 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa45834-722c-4036-a172-f507c92a8f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:04.133233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa45834-722c-4036-a172-f507c92a8f8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:04.133529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa45834-722c-4036-a172-f507c92a8f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:04.133817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa45834-722c-4036-a172-f507c92a8f8e admin admin] PATH_INFO: `/v3/projects/f4eee425606941209f6ea7b27054a448/users/441bfaaf1b344b77be4f29567fe39fb4/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:04.137546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa45834-722c-4036-a172-f507c92a8f8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4eee425606941209f6ea7b27054a448, user_id=441bfaaf1b344b77be4f29567fe39fb4, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:04.140088 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa45834-722c-4036-a172-f507c92a8f8e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:04.167788 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 947/1898] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:04 2022] PUT /identity/v3/projects/f4eee425606941209f6ea7b27054a448/users/441bfaaf1b344b77be4f29567fe39fb4/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:04.180622 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cbfc8-ab08-4e91-8169-fef64097fcef None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:04.181272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cbfc8-ab08-4e91-8169-fef64097fcef None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:04.182864 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cbfc8-ab08-4e91-8169-fef64097fcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:04.216261 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-fe2cbfc8-ab08-4e91-8169-fef64097fcef None None] MFA Rules not processed for user `441bfaaf1b344b77be4f29567fe39fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:04.218049 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fe2cbfc8-ab08-4e91-8169-fef64097fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:04.238134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e646b86-2229-4e91-8358-0cf2c3090a4b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:04.290084 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:04.321577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c70c11-0201-4a22-8a9a-660f6b19032a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:04.322238 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c70c11-0201-4a22-8a9a-660f6b19032a service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:04.322526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c70c11-0201-4a22-8a9a-660f6b19032a service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:04.323036 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c70c11-0201-4a22-8a9a-660f6b19032a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:04.333760 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 948/1899] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:04.348069 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cbfc8-ab08-4e91-8169-fef64097fcef None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:04.349430 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fe2cbfc8-ab08-4e91-8169-fef64097fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:04.361875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cbfc8-ab08-4e91-8169-fef64097fcef None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:04.365780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17c2b5-2f31-4299-9956-5a67d1cac9e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:04.366436 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17c2b5-2f31-4299-9956-5a67d1cac9e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:04.366727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17c2b5-2f31-4299-9956-5a67d1cac9e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:04.367007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17c2b5-2f31-4299-9956-5a67d1cac9e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:04.374613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:04.377302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17c2b5-2f31-4299-9956-5a67d1cac9e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:04.379052 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17c2b5-2f31-4299-9956-5a67d1cac9e3 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:04.390812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 949/1900] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:08:04.396216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c70c11-0201-4a22-8a9a-660f6b19032a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:04.397560 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c70c11-0201-4a22-8a9a-660f6b19032a service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:04.408460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 952/1901] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:04 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:08:04.434603 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17c2b5-2f31-4299-9956-5a67d1cac9e3 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:04.443487 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17c2b5-2f31-4299-9956-5a67d1cac9e3 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:04.447465 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bed4e9-2576-47bd-918c-e7bd8dbdf4a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:04.448189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bed4e9-2576-47bd-918c-e7bd8dbdf4a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:04.448506 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bed4e9-2576-47bd-918c-e7bd8dbdf4a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:04.448797 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bed4e9-2576-47bd-918c-e7bd8dbdf4a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:04.514453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bed4e9-2576-47bd-918c-e7bd8dbdf4a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:04.515996 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bed4e9-2576-47bd-918c-e7bd8dbdf4a8 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:04.527359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 950/1902] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:08.864388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6672b-fecc-4bdd-a3d1-befe818974d4 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:08.865073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6672b-fecc-4bdd-a3d1-befe818974d4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:08.865073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6672b-fecc-4bdd-a3d1-befe818974d4 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:08.867152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 953/1903] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 11:08:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:08:08.889134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853edc32-e6d6-4d3e-be4d-9885243cfc56 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:08.889668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853edc32-e6d6-4d3e-be4d-9885243cfc56 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:08.889668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853edc32-e6d6-4d3e-be4d-9885243cfc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:08.950583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-853edc32-e6d6-4d3e-be4d-9885243cfc56 None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:08.952075 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-853edc32-e6d6-4d3e-be4d-9885243cfc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:09.128695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 951/1904] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 11:08:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:09.133478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6672b-fecc-4bdd-a3d1-befe818974d4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:09.276836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853edc32-e6d6-4d3e-be4d-9885243cfc56 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:09.290182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6672b-fecc-4bdd-a3d1-befe818974d4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:09.293364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853edc32-e6d6-4d3e-be4d-9885243cfc56 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:09.296664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:09.297595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:09.297879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:09.298183 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] PATH_INFO: `/v3/users/d6b236f644b0405e8375b00e49d735ac` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:09.298789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a24f4e-a05a-48cc-9aae-3e78504b0980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:09.299555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b236f644b0405e8375b00e49d735ac)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:09.299938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a24f4e-a05a-48cc-9aae-3e78504b0980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:09.300278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a24f4e-a05a-48cc-9aae-3e78504b0980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:09.300547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a24f4e-a05a-48cc-9aae-3e78504b0980 admin admin] PATH_INFO: `/v3/users/c609de2bb6ca498ca2c0250b8c3e4f56` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:09.301800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:09.301915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a24f4e-a05a-48cc-9aae-3e78504b0980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c609de2bb6ca498ca2c0250b8c3e4f56)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:09.304094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a24f4e-a05a-48cc-9aae-3e78504b0980 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:09.353684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b236f644b0405e8375b00e49d735ac'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:09.358754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b7a24f4e-a05a-48cc-9aae-3e78504b0980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c609de2bb6ca498ca2c0250b8c3e4f56'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:09.359449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b7a24f4e-a05a-48cc-9aae-3e78504b0980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c609de2bb6ca498ca2c0250b8c3e4f56'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:09.363077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b236f644b0405e8375b00e49d735ac'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:09.363817 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b236f644b0405e8375b00e49d735ac'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:09.366921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b7a24f4e-a05a-48cc-9aae-3e78504b0980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c609de2bb6ca498ca2c0250b8c3e4f56'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:09.372009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b236f644b0405e8375b00e49d735ac'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:09.372739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b236f644b0405e8375b00e49d735ac'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:09.376462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b7a24f4e-a05a-48cc-9aae-3e78504b0980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c609de2bb6ca498ca2c0250b8c3e4f56'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:09.377118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b7a24f4e-a05a-48cc-9aae-3e78504b0980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c609de2bb6ca498ca2c0250b8c3e4f56'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:09.390838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:09.392653 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 954/1905] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:09 2022] DELETE /identity/v3/users/d6b236f644b0405e8375b00e49d735ac => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:09.395968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:09.395968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:09.403337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 952/1906] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:09 2022] DELETE /identity/v3/users/c609de2bb6ca498ca2c0250b8c3e4f56 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:09.403337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e73654-ad28-40b8-8a70-63c98fb4db3a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:09.403337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e73654-ad28-40b8-8a70-63c98fb4db3a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:09.403337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e73654-ad28-40b8-8a70-63c98fb4db3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:09.413829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:09.426307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-30e73654-ad28-40b8-8a70-63c98fb4db3a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:09.427572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-30e73654-ad28-40b8-8a70-63c98fb4db3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:09.487604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dd87b-f425-4a87-aa4e-67a128e08f9e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:09.500319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 953/1907] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:09.501941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e57593-d35b-486d-ba03-c85688cd0bf2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:09.502310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e57593-d35b-486d-ba03-c85688cd0bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:09.502651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e57593-d35b-486d-ba03-c85688cd0bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:09.521034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-32e57593-d35b-486d-ba03-c85688cd0bf2 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:09.522283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-32e57593-d35b-486d-ba03-c85688cd0bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:09.530978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 954/1908] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:09.531851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e57593-d35b-486d-ba03-c85688cd0bf2 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:09.533198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-32e57593-d35b-486d-ba03-c85688cd0bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:09.563223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627635a8-813c-472e-a33d-9b41af75555d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:09.564261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627635a8-813c-472e-a33d-9b41af75555d service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:09.564728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627635a8-813c-472e-a33d-9b41af75555d service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:09.565071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627635a8-813c-472e-a33d-9b41af75555d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:09.567289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-627635a8-813c-472e-a33d-9b41af75555d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:09.584479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627635a8-813c-472e-a33d-9b41af75555d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:09.586477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627635a8-813c-472e-a33d-9b41af75555d service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:09.598975 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 955/1909] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:09.599835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627635a8-813c-472e-a33d-9b41af75555d service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:09.601694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-627635a8-813c-472e-a33d-9b41af75555d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:09.618162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627635a8-813c-472e-a33d-9b41af75555d service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:09.618650 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e57593-d35b-486d-ba03-c85688cd0bf2 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:09.622362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44746725-ae2a-452f-8e92-3d615b009811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:09.622743 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c472f58-6a16-43a8-8371-7e2a1a1dcf6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:09.623021 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44746725-ae2a-452f-8e92-3d615b009811 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:09.623305 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44746725-ae2a-452f-8e92-3d615b009811 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:09.623463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c472f58-6a16-43a8-8371-7e2a1a1dcf6e service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:09.623618 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44746725-ae2a-452f-8e92-3d615b009811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:09.623717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c472f58-6a16-43a8-8371-7e2a1a1dcf6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:09.624092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c472f58-6a16-43a8-8371-7e2a1a1dcf6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:09.634332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44746725-ae2a-452f-8e92-3d615b009811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:09.636233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44746725-ae2a-452f-8e92-3d615b009811 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:09.637783 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c472f58-6a16-43a8-8371-7e2a1a1dcf6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:09.640903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c472f58-6a16-43a8-8371-7e2a1a1dcf6e service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:09.649806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 956/1910] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:09.655759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 955/1911] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:09.999929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44746725-ae2a-452f-8e92-3d615b009811 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:10.001823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-44746725-ae2a-452f-8e92-3d615b009811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:10.016489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c472f58-6a16-43a8-8371-7e2a1a1dcf6e service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:10.017939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0c472f58-6a16-43a8-8371-7e2a1a1dcf6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:10.034893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44746725-ae2a-452f-8e92-3d615b009811 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:10.040574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:10.042161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:10.042514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:10.045837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] PATH_INFO: `/v3/projects/ffef335cbe6846248ed7d672d7acd424` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:10.045837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffef335cbe6846248ed7d672d7acd424)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:10.047648 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:10.062568 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c472f58-6a16-43a8-8371-7e2a1a1dcf6e service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:10.079532 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:10.080412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:10.080699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:10.081132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] PATH_INFO: `/v3/projects/3d1dbd18c98949fabf3a1dd1c1d9691a` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:10.088095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d1dbd18c98949fabf3a1dd1c1d9691a)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:10.090090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:10.187585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d1dbd18c98949fabf3a1dd1c1d9691a', 'request_id': 'req-230bff34-b187-4304-9f05-42c62f19919b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.189681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffef335cbe6846248ed7d672d7acd424', 'request_id': 'req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.197179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffef335cbe6846248ed7d672d7acd424', 'request_id': 'req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.206879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffef335cbe6846248ed7d672d7acd424', 'request_id': 'req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.214119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d1dbd18c98949fabf3a1dd1c1d9691a', 'request_id': 'req-230bff34-b187-4304-9f05-42c62f19919b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.222705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d1dbd18c98949fabf3a1dd1c1d9691a', 'request_id': 'req-230bff34-b187-4304-9f05-42c62f19919b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.228443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] The token cache is being invalidate because project ffef335cbe6846248ed7d672d7acd424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:10.228983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.230405 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.232484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 957/1912] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:09 2022] DELETE /identity/v3/projects/ffef335cbe6846248ed7d672d7acd424 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:10.248168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:10.250184 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:10.253135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] The token cache is being invalidate because project 3d1dbd18c98949fabf3a1dd1c1d9691a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:10.253441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.255890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.259437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 956/1913] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:10 2022] DELETE /identity/v3/projects/3d1dbd18c98949fabf3a1dd1c1d9691a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:10.263308 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:10.267316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:10.268189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:10.326419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:10.427440 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda4f5e-1ab6-4e3e-91fb-9bb02afd456b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:10.436131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bff34-b187-4304-9f05-42c62f19919b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:10.440463 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:10.442379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:10.442871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:10.443718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] PATH_INFO: `/v3/users/07f63cff45d4483ca5c66f1f1da25ccc` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:10.444399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:10.445916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:10.446426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:10.446607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f63cff45d4483ca5c66f1f1da25ccc)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:10.446923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] PATH_INFO: `/v3/users/0dbd6661f8694640b14dd9d647410b92` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:10.449855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dbd6661f8694640b14dd9d647410b92)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:10.452616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:10.453566 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:10.548509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f63cff45d4483ca5c66f1f1da25ccc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.561185 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f63cff45d4483ca5c66f1f1da25ccc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.561880 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f63cff45d4483ca5c66f1f1da25ccc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.569823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f63cff45d4483ca5c66f1f1da25ccc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.570501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f63cff45d4483ca5c66f1f1da25ccc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.573302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dbd6661f8694640b14dd9d647410b92'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.574407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dbd6661f8694640b14dd9d647410b92'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.584706 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dbd6661f8694640b14dd9d647410b92'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.589719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 958/1914] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:10 2022] DELETE /identity/v3/users/07f63cff45d4483ca5c66f1f1da25ccc => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:10.592662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dbd6661f8694640b14dd9d647410b92'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.593694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dbd6661f8694640b14dd9d647410b92'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.613417 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 957/1915] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:10 2022] DELETE /identity/v3/users/0dbd6661f8694640b14dd9d647410b92 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:11.034559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:11.036098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:11.093565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:11.095477 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:11.115609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bcee4-4707-4cb8-a8a6-47bdc740b01f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:11.155205 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:11.200559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bc85d-a8b6-4b5d-b261-d16cd9235d39 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:11.201568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fb42e-ad87-4b22-826b-742febfe4983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:11.202867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fb42e-ad87-4b22-826b-742febfe4983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.203284 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fb42e-ad87-4b22-826b-742febfe4983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.203603 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fb42e-ad87-4b22-826b-742febfe4983 admin admin] PATH_INFO: `/v3/projects/b7f363554d98430286c6b81cb6e5d7f7` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.205148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fb42e-ad87-4b22-826b-742febfe4983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f363554d98430286c6b81cb6e5d7f7)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:11.207527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fb42e-ad87-4b22-826b-742febfe4983 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:11.209137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:11.212061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.212549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.214200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] PATH_INFO: `/v3/projects/5c6421bc17ca4436a75a4d95b9596d7f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.217255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6421bc17ca4436a75a4d95b9596d7f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:11.220703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:11.276772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-af4fb42e-ad87-4b22-826b-742febfe4983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f363554d98430286c6b81cb6e5d7f7', 'request_id': 'req-af4fb42e-ad87-4b22-826b-742febfe4983'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.283307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-af4fb42e-ad87-4b22-826b-742febfe4983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f363554d98430286c6b81cb6e5d7f7', 'request_id': 'req-af4fb42e-ad87-4b22-826b-742febfe4983'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.292398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-af4fb42e-ad87-4b22-826b-742febfe4983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f363554d98430286c6b81cb6e5d7f7', 'request_id': 'req-af4fb42e-ad87-4b22-826b-742febfe4983'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.297347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:11.314119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-af4fb42e-ad87-4b22-826b-742febfe4983 admin admin] The token cache is being invalidate because project b7f363554d98430286c6b81cb6e5d7f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:11.314119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-af4fb42e-ad87-4b22-826b-742febfe4983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.314119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-af4fb42e-ad87-4b22-826b-742febfe4983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.316022 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 959/1916] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:11 2022] DELETE /identity/v3/projects/b7f363554d98430286c6b81cb6e5d7f7 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:11.322127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6421bc17ca4436a75a4d95b9596d7f', 'request_id': 'req-3d8c9492-cbfe-48f6-a708-52a96993b882'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.323751 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fb408-8fc2-4965-ae6d-201d1d46240d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.324403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fb408-8fc2-4965-ae6d-201d1d46240d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.324795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fb408-8fc2-4965-ae6d-201d1d46240d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.332048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6421bc17ca4436a75a4d95b9596d7f', 'request_id': 'req-3d8c9492-cbfe-48f6-a708-52a96993b882'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.340844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6421bc17ca4436a75a4d95b9596d7f', 'request_id': 'req-3d8c9492-cbfe-48f6-a708-52a96993b882'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.355657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bd4fb408-8fc2-4965-ae6d-201d1d46240d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:11.359339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bd4fb408-8fc2-4965-ae6d-201d1d46240d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:11.361560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] The token cache is being invalidate because project 5c6421bc17ca4436a75a4d95b9596d7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:11.362300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.363184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.365406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 958/1917] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:11 2022] DELETE /identity/v3/projects/5c6421bc17ca4436a75a4d95b9596d7f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:11.369084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:11.371219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:11.462808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 960/1918] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:11.464868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0758ef-325b-4d4b-b148-0e1feb20d96c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.465159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0758ef-325b-4d4b-b148-0e1feb20d96c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.471715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0758ef-325b-4d4b-b148-0e1feb20d96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.478090 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c9492-cbfe-48f6-a708-52a96993b882 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:11.486233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:11.488089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.488089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.488089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] PATH_INFO: `/v3/users/441bfaaf1b344b77be4f29567fe39fb4` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.489321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] RBAC: Authorizing `identity:delete_user(user_id=441bfaaf1b344b77be4f29567fe39fb4)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:11.492019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:11.502421 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fd0758ef-325b-4d4b-b148-0e1feb20d96c None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:11.504617 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fd0758ef-325b-4d4b-b148-0e1feb20d96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:11.523841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 961/1919] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:11.525139 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0758ef-325b-4d4b-b148-0e1feb20d96c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:11.547874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0758ef-325b-4d4b-b148-0e1feb20d96c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:11.556046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a7ae0-eac4-49fe-b254-87bf0b5d8506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:11.556046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a7ae0-eac4-49fe-b254-87bf0b5d8506 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.556046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a7ae0-eac4-49fe-b254-87bf0b5d8506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.556046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a7ae0-eac4-49fe-b254-87bf0b5d8506 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.557659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a7ae0-eac4-49fe-b254-87bf0b5d8506 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:11.557659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a7ae0-eac4-49fe-b254-87bf0b5d8506 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:11.558909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a7ae0-eac4-49fe-b254-87bf0b5d8506 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:11.577783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 962/1920] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:08:11.578723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a7ae0-eac4-49fe-b254-87bf0b5d8506 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:11.591681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a7ae0-eac4-49fe-b254-87bf0b5d8506 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:11.597269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '441bfaaf1b344b77be4f29567fe39fb4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.598358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5440c5-6a3e-4c77-9651-2fc911d56efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:11.599044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5440c5-6a3e-4c77-9651-2fc911d56efd admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.599332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5440c5-6a3e-4c77-9651-2fc911d56efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.599646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5440c5-6a3e-4c77-9651-2fc911d56efd admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.599938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '441bfaaf1b344b77be4f29567fe39fb4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.600342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5440c5-6a3e-4c77-9651-2fc911d56efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:11.602640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5440c5-6a3e-4c77-9651-2fc911d56efd admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:11.613908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '441bfaaf1b344b77be4f29567fe39fb4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.626883 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '441bfaaf1b344b77be4f29567fe39fb4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.630711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '441bfaaf1b344b77be4f29567fe39fb4'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.645660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 963/1921] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:11 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:11.651696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5440c5-6a3e-4c77-9651-2fc911d56efd admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:11.654180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5e5440c5-6a3e-4c77-9651-2fc911d56efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:11.669221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 959/1922] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:11 2022] DELETE /identity/v3/users/441bfaaf1b344b77be4f29567fe39fb4 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:11.685921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:11.752731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5440c5-6a3e-4c77-9651-2fc911d56efd admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:11.859798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d095773-3f64-4d7e-bbc1-7d6d9d37a713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:11.860582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d095773-3f64-4d7e-bbc1-7d6d9d37a713 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.860946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d095773-3f64-4d7e-bbc1-7d6d9d37a713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.861295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d095773-3f64-4d7e-bbc1-7d6d9d37a713 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.862005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d095773-3f64-4d7e-bbc1-7d6d9d37a713 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:11.864067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d095773-3f64-4d7e-bbc1-7d6d9d37a713 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:11.887978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 964/1923] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:11 2022] POST /identity/v3/users => generated 291 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:11.892125 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:11.909247 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563b409-f5bc-40ea-8578-6e4a4af2394e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:11.916222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e400c3-5cc6-428e-bb3d-8988867d69e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:11.917606 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e400c3-5cc6-428e-bb3d-8988867d69e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.918031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e400c3-5cc6-428e-bb3d-8988867d69e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.920544 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e400c3-5cc6-428e-bb3d-8988867d69e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.920544 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e400c3-5cc6-428e-bb3d-8988867d69e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:11.921288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e400c3-5cc6-428e-bb3d-8988867d69e1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:11.935201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 960/1924] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:11.941149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d095773-3f64-4d7e-bbc1-7d6d9d37a713 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:11.958226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d095773-3f64-4d7e-bbc1-7d6d9d37a713 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:11.967353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac233db-f76d-49bf-bf39-5c76a0bc5a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:11.968704 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac233db-f76d-49bf-bf39-5c76a0bc5a96 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.970735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac233db-f76d-49bf-bf39-5c76a0bc5a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.971244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac233db-f76d-49bf-bf39-5c76a0bc5a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.972740 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac233db-f76d-49bf-bf39-5c76a0bc5a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:11.975895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac233db-f76d-49bf-bf39-5c76a0bc5a96 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:11.996451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 965/1925] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:12.002228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e400c3-5cc6-428e-bb3d-8988867d69e1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.014281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e400c3-5cc6-428e-bb3d-8988867d69e1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.019331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a4735-273d-489c-8bdb-1f61c3186b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.020356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a4735-273d-489c-8bdb-1f61c3186b81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.020715 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a4735-273d-489c-8bdb-1f61c3186b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.021087 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a4735-273d-489c-8bdb-1f61c3186b81 admin admin] PATH_INFO: `/v3/projects/3e7500e9c69444c2a10857432d92688f/users/f3b81904965341b19b7221220a55cee3/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.039562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a4735-273d-489c-8bdb-1f61c3186b81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e7500e9c69444c2a10857432d92688f, user_id=f3b81904965341b19b7221220a55cee3, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.042348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a4735-273d-489c-8bdb-1f61c3186b81 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.057622 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 961/1926] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:12 2022] PUT /identity/v3/projects/3e7500e9c69444c2a10857432d92688f/users/f3b81904965341b19b7221220a55cee3/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:12.068806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba097a61-0626-469b-a36a-95097107d957 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.069452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba097a61-0626-469b-a36a-95097107d957 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.069452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba097a61-0626-469b-a36a-95097107d957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.085687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a4735-273d-489c-8bdb-1f61c3186b81 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.087902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-595a4735-273d-489c-8bdb-1f61c3186b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:12.107168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:12.117518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ba097a61-0626-469b-a36a-95097107d957 None None] MFA Rules not processed for user `f3b81904965341b19b7221220a55cee3`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:12.119397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ba097a61-0626-469b-a36a-95097107d957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:12.182172 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a4735-273d-489c-8bdb-1f61c3186b81 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.186889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb3311-785f-4388-8714-bda7ae9a0141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.187793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb3311-785f-4388-8714-bda7ae9a0141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.188129 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb3311-785f-4388-8714-bda7ae9a0141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.188415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb3311-785f-4388-8714-bda7ae9a0141 admin admin] PATH_INFO: `/v3/projects/f4eee425606941209f6ea7b27054a448` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.189626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb3311-785f-4388-8714-bda7ae9a0141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4eee425606941209f6ea7b27054a448)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.191594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb3311-785f-4388-8714-bda7ae9a0141 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.208360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 966/1927] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:12.212784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba097a61-0626-469b-a36a-95097107d957 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.226279 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba097a61-0626-469b-a36a-95097107d957 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.234091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445c3ef-39c1-49eb-9dfa-05528c9b2af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.234811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445c3ef-39c1-49eb-9dfa-05528c9b2af9 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.235098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445c3ef-39c1-49eb-9dfa-05528c9b2af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.235400 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445c3ef-39c1-49eb-9dfa-05528c9b2af9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.236107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445c3ef-39c1-49eb-9dfa-05528c9b2af9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.239176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445c3ef-39c1-49eb-9dfa-05528c9b2af9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.261034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-efbb3311-785f-4388-8714-bda7ae9a0141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4eee425606941209f6ea7b27054a448', 'request_id': 'req-efbb3311-785f-4388-8714-bda7ae9a0141'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:12.268566 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-efbb3311-785f-4388-8714-bda7ae9a0141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4eee425606941209f6ea7b27054a448', 'request_id': 'req-efbb3311-785f-4388-8714-bda7ae9a0141'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:12.277775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-efbb3311-785f-4388-8714-bda7ae9a0141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4eee425606941209f6ea7b27054a448', 'request_id': 'req-efbb3311-785f-4388-8714-bda7ae9a0141'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:12.280407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 967/1928] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:12 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:08:12.284455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445c3ef-39c1-49eb-9dfa-05528c9b2af9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.299141 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:12.303221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-efbb3311-785f-4388-8714-bda7ae9a0141 admin admin] The token cache is being invalidate because project f4eee425606941209f6ea7b27054a448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:12.303687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-efbb3311-785f-4388-8714-bda7ae9a0141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:12.304733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-efbb3311-785f-4388-8714-bda7ae9a0141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:12.306783 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 962/1929] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:12 2022] DELETE /identity/v3/projects/f4eee425606941209f6ea7b27054a448 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:12.312896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa8643-c798-4336-9e61-aea153261d17 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.313190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa8643-c798-4336-9e61-aea153261d17 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.313470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa8643-c798-4336-9e61-aea153261d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.335154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-34aa8643-c798-4336-9e61-aea153261d17 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:12.336399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-34aa8643-c798-4336-9e61-aea153261d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:12.364714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445c3ef-39c1-49eb-9dfa-05528c9b2af9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.381067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8f19a-d22a-4f2c-91df-a664904c2c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.381770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8f19a-d22a-4f2c-91df-a664904c2c62 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.382067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8f19a-d22a-4f2c-91df-a664904c2c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.384651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8f19a-d22a-4f2c-91df-a664904c2c62 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.384651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8f19a-d22a-4f2c-91df-a664904c2c62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.385807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8f19a-d22a-4f2c-91df-a664904c2c62 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.423413 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 963/1930] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:12.423806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 968/1931] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:12 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:08:12.424075 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa8643-c798-4336-9e61-aea153261d17 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.428732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-34aa8643-c798-4336-9e61-aea153261d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:12.428907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52b303-8253-4eaa-8f4f-46387a108367 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.428907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52b303-8253-4eaa-8f4f-46387a108367 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.428907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52b303-8253-4eaa-8f4f-46387a108367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.448623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa8643-c798-4336-9e61-aea153261d17 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.455664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e7678-aa23-4de2-a728-76527d068dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.456748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e7678-aa23-4de2-a728-76527d068dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.457215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e7678-aa23-4de2-a728-76527d068dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.457677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e7678-aa23-4de2-a728-76527d068dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.458628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4e52b303-8253-4eaa-8f4f-46387a108367 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:12.458925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e7678-aa23-4de2-a728-76527d068dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.460569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4e52b303-8253-4eaa-8f4f-46387a108367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:12.462202 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e7678-aa23-4de2-a728-76527d068dc6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.475430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 969/1932] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:12.477383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52b303-8253-4eaa-8f4f-46387a108367 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.479979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 964/1933] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:12.484149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e7678-aa23-4de2-a728-76527d068dc6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.487170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52b303-8253-4eaa-8f4f-46387a108367 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.492364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f208e3-88eb-41ca-a5d6-a7ab2e5dc73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.493053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f208e3-88eb-41ca-a5d6-a7ab2e5dc73f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.493340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f208e3-88eb-41ca-a5d6-a7ab2e5dc73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.493620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f208e3-88eb-41ca-a5d6-a7ab2e5dc73f admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.494888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f208e3-88eb-41ca-a5d6-a7ab2e5dc73f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:12.495196 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f208e3-88eb-41ca-a5d6-a7ab2e5dc73f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.495457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e7678-aa23-4de2-a728-76527d068dc6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.496847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f208e3-88eb-41ca-a5d6-a7ab2e5dc73f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.507454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bfb44-767a-42e0-bf8f-39e05eb3e645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.508501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bfb44-767a-42e0-bf8f-39e05eb3e645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.508851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bfb44-767a-42e0-bf8f-39e05eb3e645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.509237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bfb44-767a-42e0-bf8f-39e05eb3e645 admin admin] PATH_INFO: `/v3/projects/b41603209cd140d886232bc729a17e41/users/584f515aeb2d419c8e8cabb406269297/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.509690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 970/1934] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:08:12.515571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f208e3-88eb-41ca-a5d6-a7ab2e5dc73f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.526500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f208e3-88eb-41ca-a5d6-a7ab2e5dc73f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.527668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bfb44-767a-42e0-bf8f-39e05eb3e645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41603209cd140d886232bc729a17e41, user_id=584f515aeb2d419c8e8cabb406269297, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.530689 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bfb44-767a-42e0-bf8f-39e05eb3e645 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.531656 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80284755-1bc7-4621-a776-bc572d3eec49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.532538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80284755-1bc7-4621-a776-bc572d3eec49 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.532995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80284755-1bc7-4621-a776-bc572d3eec49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.533509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80284755-1bc7-4621-a776-bc572d3eec49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.535301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80284755-1bc7-4621-a776-bc572d3eec49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.538988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80284755-1bc7-4621-a776-bc572d3eec49 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.555360 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 965/1935] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:12 2022] PUT /identity/v3/projects/b41603209cd140d886232bc729a17e41/users/584f515aeb2d419c8e8cabb406269297/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:12.559201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bfb44-767a-42e0-bf8f-39e05eb3e645 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.585570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 971/1936] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:12 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:12.585570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80284755-1bc7-4621-a776-bc572d3eec49 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.588650 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:12.697016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80284755-1bc7-4621-a776-bc572d3eec49 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.703121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f8310-f323-4899-bff9-65ad047197bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.703860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f8310-f323-4899-bff9-65ad047197bf admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.704580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f8310-f323-4899-bff9-65ad047197bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.704580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f8310-f323-4899-bff9-65ad047197bf admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.705115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f8310-f323-4899-bff9-65ad047197bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.707235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f8310-f323-4899-bff9-65ad047197bf admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.729370 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bfb44-767a-42e0-bf8f-39e05eb3e645 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.736757 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87eb97-7f6f-41c1-a14c-79dfbd524e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.737450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87eb97-7f6f-41c1-a14c-79dfbd524e51 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.737741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87eb97-7f6f-41c1-a14c-79dfbd524e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.738150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87eb97-7f6f-41c1-a14c-79dfbd524e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.738899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87eb97-7f6f-41c1-a14c-79dfbd524e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.741120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87eb97-7f6f-41c1-a14c-79dfbd524e51 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.742246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 972/1937] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:12 2022] POST /identity/v3/users => generated 291 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:12.743214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f8310-f323-4899-bff9-65ad047197bf admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.755649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 966/1938] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:08:12.756512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f8310-f323-4899-bff9-65ad047197bf admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.760494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87eb97-7f6f-41c1-a14c-79dfbd524e51 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.765427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eafb70-85e7-4f3b-ac77-3e483d235910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.766136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eafb70-85e7-4f3b-ac77-3e483d235910 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.766419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eafb70-85e7-4f3b-ac77-3e483d235910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.766705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eafb70-85e7-4f3b-ac77-3e483d235910 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.770202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eafb70-85e7-4f3b-ac77-3e483d235910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.771887 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eafb70-85e7-4f3b-ac77-3e483d235910 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.778646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87eb97-7f6f-41c1-a14c-79dfbd524e51 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.787169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 973/1939] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:12.788236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eafb70-85e7-4f3b-ac77-3e483d235910 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.790659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a90e7-0fb5-42a4-b4bd-f301af6e4069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.792448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a90e7-0fb5-42a4-b4bd-f301af6e4069 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.793191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a90e7-0fb5-42a4-b4bd-f301af6e4069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.793488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a90e7-0fb5-42a4-b4bd-f301af6e4069 admin admin] PATH_INFO: `/v3/projects/b41603209cd140d886232bc729a17e41/users/584f515aeb2d419c8e8cabb406269297/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.796797 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a90e7-0fb5-42a4-b4bd-f301af6e4069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41603209cd140d886232bc729a17e41, user_id=584f515aeb2d419c8e8cabb406269297, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.800050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eafb70-85e7-4f3b-ac77-3e483d235910 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.800205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a90e7-0fb5-42a4-b4bd-f301af6e4069 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.805914 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a5f12-292c-4fa0-b181-dfafb1d5ad13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.806615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a5f12-292c-4fa0-b181-dfafb1d5ad13 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.808274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a5f12-292c-4fa0-b181-dfafb1d5ad13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.808274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a5f12-292c-4fa0-b181-dfafb1d5ad13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.808651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a5f12-292c-4fa0-b181-dfafb1d5ad13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.810420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a5f12-292c-4fa0-b181-dfafb1d5ad13 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.818276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 967/1940] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:12 2022] PUT /identity/v3/projects/b41603209cd140d886232bc729a17e41/users/584f515aeb2d419c8e8cabb406269297/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:12.830539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfcbff-45cc-4e71-bc2f-9654724118c6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.831296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfcbff-45cc-4e71-bc2f-9654724118c6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.831678 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfcbff-45cc-4e71-bc2f-9654724118c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.831862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 974/1941] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:12.833452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a5f12-292c-4fa0-b181-dfafb1d5ad13 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.844909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a5f12-292c-4fa0-b181-dfafb1d5ad13 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.872746 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-55bfcbff-45cc-4e71-bc2f-9654724118c6 None None] MFA Rules not processed for user `584f515aeb2d419c8e8cabb406269297`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:12.875042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-55bfcbff-45cc-4e71-bc2f-9654724118c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:12.876239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:12.939261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52bdc02-c371-45d5-b0ac-05e6f856eb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.940372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bdc02-c371-45d5-b0ac-05e6f856eb27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.940920 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bdc02-c371-45d5-b0ac-05e6f856eb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.941253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bdc02-c371-45d5-b0ac-05e6f856eb27 admin admin] PATH_INFO: `/v3/projects/c902cd4854314a84b311d5a5c8b0927d/users/1575c0bf84fd4261bbd10ea33dfc67f6/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.974316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52bdc02-c371-45d5-b0ac-05e6f856eb27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c902cd4854314a84b311d5a5c8b0927d, user_id=1575c0bf84fd4261bbd10ea33dfc67f6, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.977094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52bdc02-c371-45d5-b0ac-05e6f856eb27 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.999557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 975/1942] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:12 2022] PUT /identity/v3/projects/c902cd4854314a84b311d5a5c8b0927d/users/1575c0bf84fd4261bbd10ea33dfc67f6/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:13.005702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 968/1943] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:13.018581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfcbff-45cc-4e71-bc2f-9654724118c6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.019988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0aec3-b5a8-4737-939d-9cc6fb733463 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.020276 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0aec3-b5a8-4737-939d-9cc6fb733463 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.020554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0aec3-b5a8-4737-939d-9cc6fb733463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.022844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-55bfcbff-45cc-4e71-bc2f-9654724118c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:13.072048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e8b0aec3-b5a8-4737-939d-9cc6fb733463 None None] MFA Rules not processed for user `1575c0bf84fd4261bbd10ea33dfc67f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:13.073811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e8b0aec3-b5a8-4737-939d-9cc6fb733463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:13.096059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfcbff-45cc-4e71-bc2f-9654724118c6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.131942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:13.154360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 976/1944] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:13.157207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfab52-61cf-43b4-b1ef-c93e3acb1af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.158029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfab52-61cf-43b4-b1ef-c93e3acb1af4 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.158455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfab52-61cf-43b4-b1ef-c93e3acb1af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.158756 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0aec3-b5a8-4737-939d-9cc6fb733463 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.158953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfab52-61cf-43b4-b1ef-c93e3acb1af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.244958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0aec3-b5a8-4737-939d-9cc6fb733463 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.250778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b1179-4edc-481c-9fba-fa0d01cf2869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.251503 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b1179-4edc-481c-9fba-fa0d01cf2869 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.251794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b1179-4edc-481c-9fba-fa0d01cf2869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.252074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b1179-4edc-481c-9fba-fa0d01cf2869 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.252183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bfab52-61cf-43b4-b1ef-c93e3acb1af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.252775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b1179-4edc-481c-9fba-fa0d01cf2869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.253635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bfab52-61cf-43b4-b1ef-c93e3acb1af4 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.254703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b1179-4edc-481c-9fba-fa0d01cf2869 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.268170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 969/1945] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:13.292683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 977/1946] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:13 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:08:13.296634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfab52-61cf-43b4-b1ef-c93e3acb1af4 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.311359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfab52-61cf-43b4-b1ef-c93e3acb1af4 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.384519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b1179-4edc-481c-9fba-fa0d01cf2869 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.387591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a78b1179-4edc-481c-9fba-fa0d01cf2869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:13.389508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951913d-f735-434f-ad7c-07b490d3ee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.390207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951913d-f735-434f-ad7c-07b490d3ee10 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.390640 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951913d-f735-434f-ad7c-07b490d3ee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.390939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951913d-f735-434f-ad7c-07b490d3ee10 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.393850 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951913d-f735-434f-ad7c-07b490d3ee10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.395937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951913d-f735-434f-ad7c-07b490d3ee10 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.425767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 970/1947] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:13 2022] POST /identity/v3/users => generated 298 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:13.427228 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:13.436467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951913d-f735-434f-ad7c-07b490d3ee10 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.455096 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951913d-f735-434f-ad7c-07b490d3ee10 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.468226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9383254-18e2-4c45-b7c6-31a2049bff31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.469000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9383254-18e2-4c45-b7c6-31a2049bff31 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.469350 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9383254-18e2-4c45-b7c6-31a2049bff31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.469702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9383254-18e2-4c45-b7c6-31a2049bff31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.470613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9383254-18e2-4c45-b7c6-31a2049bff31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.472448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9383254-18e2-4c45-b7c6-31a2049bff31 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.473264 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b1179-4edc-481c-9fba-fa0d01cf2869 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.477695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98781dc6-36a7-4099-89aa-aab03a72e9a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.478366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98781dc6-36a7-4099-89aa-aab03a72e9a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.479286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98781dc6-36a7-4099-89aa-aab03a72e9a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.479691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98781dc6-36a7-4099-89aa-aab03a72e9a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.481862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-98781dc6-36a7-4099-89aa-aab03a72e9a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:13.487499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 971/1948] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:13.492593 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9383254-18e2-4c45-b7c6-31a2049bff31 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.503111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9383254-18e2-4c45-b7c6-31a2049bff31 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.507995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1af30-e12c-435e-b636-562c5c6e9df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.508901 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1af30-e12c-435e-b636-562c5c6e9df5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.509184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1af30-e12c-435e-b636-562c5c6e9df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.509460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1af30-e12c-435e-b636-562c5c6e9df5 admin admin] PATH_INFO: `/v3/projects/d31c6625e6d046b1b4fdd19fd8b6dc2b/users/2ed4fcaed5804cf7917a122cf8234a52/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.532242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1af30-e12c-435e-b636-562c5c6e9df5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d31c6625e6d046b1b4fdd19fd8b6dc2b, user_id=2ed4fcaed5804cf7917a122cf8234a52, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.535556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1af30-e12c-435e-b636-562c5c6e9df5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.552513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 972/1949] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:13 2022] PUT /identity/v3/projects/d31c6625e6d046b1b4fdd19fd8b6dc2b/users/2ed4fcaed5804cf7917a122cf8234a52/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:13.556472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1af30-e12c-435e-b636-562c5c6e9df5 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.561052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98781dc6-36a7-4099-89aa-aab03a72e9a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.564411 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98781dc6-36a7-4099-89aa-aab03a72e9a2 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.574187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 978/1950] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:13.620597 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:13.664933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1af30-e12c-435e-b636-562c5c6e9df5 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.670826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9b723-5b18-4662-90aa-7f040a7d8b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.672271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9b723-5b18-4662-90aa-7f040a7d8b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.672271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9b723-5b18-4662-90aa-7f040a7d8b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.672271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9b723-5b18-4662-90aa-7f040a7d8b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.673286 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9b723-5b18-4662-90aa-7f040a7d8b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.675126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9b723-5b18-4662-90aa-7f040a7d8b2b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.698970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 973/1951] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:08:13.703124 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98781dc6-36a7-4099-89aa-aab03a72e9a2 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.717076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98781dc6-36a7-4099-89aa-aab03a72e9a2 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.722454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c7a14-0ab9-4029-b089-75b7a0f334a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.723507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c7a14-0ab9-4029-b089-75b7a0f334a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.723852 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c7a14-0ab9-4029-b089-75b7a0f334a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.724323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c7a14-0ab9-4029-b089-75b7a0f334a5 admin admin] PATH_INFO: `/v3/projects/d31c6625e6d046b1b4fdd19fd8b6dc2b/users/2ed4fcaed5804cf7917a122cf8234a52/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.726604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c7a14-0ab9-4029-b089-75b7a0f334a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d31c6625e6d046b1b4fdd19fd8b6dc2b, user_id=2ed4fcaed5804cf7917a122cf8234a52, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.729446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c7a14-0ab9-4029-b089-75b7a0f334a5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.751070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 979/1952] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:13 2022] PUT /identity/v3/projects/d31c6625e6d046b1b4fdd19fd8b6dc2b/users/2ed4fcaed5804cf7917a122cf8234a52/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:13.762860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1019c-b819-4654-83df-d5135a3b34cd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.763192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1019c-b819-4654-83df-d5135a3b34cd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.763484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1019c-b819-4654-83df-d5135a3b34cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.796437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-74a1019c-b819-4654-83df-d5135a3b34cd None None] MFA Rules not processed for user `2ed4fcaed5804cf7917a122cf8234a52`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:13.798553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-74a1019c-b819-4654-83df-d5135a3b34cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:13.919074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 974/1953] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:13.938012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c7a14-0ab9-4029-b089-75b7a0f334a5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:14.104096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c7a14-0ab9-4029-b089-75b7a0f334a5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:14.114768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15883b2-a2b2-4aed-86e1-07fe498862dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:14.114768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15883b2-a2b2-4aed-86e1-07fe498862dc service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:14.114768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15883b2-a2b2-4aed-86e1-07fe498862dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:14.114768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15883b2-a2b2-4aed-86e1-07fe498862dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:14.129102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15883b2-a2b2-4aed-86e1-07fe498862dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:14.132120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15883b2-a2b2-4aed-86e1-07fe498862dc service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:14.149081 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 980/1954] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:14.253533 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1019c-b819-4654-83df-d5135a3b34cd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:14.265219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1019c-b819-4654-83df-d5135a3b34cd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:14.269013 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903c715-b70d-4cf7-b985-03fb485de77f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:14.269679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903c715-b70d-4cf7-b985-03fb485de77f service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:14.269970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903c715-b70d-4cf7-b985-03fb485de77f service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:14.270256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903c715-b70d-4cf7-b985-03fb485de77f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:14.338698 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903c715-b70d-4cf7-b985-03fb485de77f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:14.340172 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903c715-b70d-4cf7-b985-03fb485de77f service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:14.349443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 975/1955] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:14.641618 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15883b2-a2b2-4aed-86e1-07fe498862dc service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:14.708165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15883b2-a2b2-4aed-86e1-07fe498862dc service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:14.712746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:14.714179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:14.714484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:14.714764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] PATH_INFO: `/v3/users/1575c0bf84fd4261bbd10ea33dfc67f6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:14.715995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1575c0bf84fd4261bbd10ea33dfc67f6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:14.717978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:14.730895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:14.772112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1575c0bf84fd4261bbd10ea33dfc67f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.781394 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1575c0bf84fd4261bbd10ea33dfc67f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.782152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1575c0bf84fd4261bbd10ea33dfc67f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.789974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1575c0bf84fd4261bbd10ea33dfc67f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.790685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1575c0bf84fd4261bbd10ea33dfc67f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.809175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 981/1956] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:14 2022] DELETE /identity/v3/users/1575c0bf84fd4261bbd10ea33dfc67f6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:14.814489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a9a2e-c34b-4f17-9dd7-783f9cad3d7d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:14.814781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a9a2e-c34b-4f17-9dd7-783f9cad3d7d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:14.815065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a9a2e-c34b-4f17-9dd7-783f9cad3d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:14.836584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-951a9a2e-c34b-4f17-9dd7-783f9cad3d7d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:14.837859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-951a9a2e-c34b-4f17-9dd7-783f9cad3d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:14.890367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:14.905131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 976/1957] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:14.915569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:14.918785 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:14.978436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a1a7e-f8b3-4504-9efe-5952a493234c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:14.982806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b9633-1817-4570-8f54-d8a20383c5cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:14.983503 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b9633-1817-4570-8f54-d8a20383c5cb service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:14.983795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b9633-1817-4570-8f54-d8a20383c5cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:14.984081 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b9633-1817-4570-8f54-d8a20383c5cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:14.994318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b9633-1817-4570-8f54-d8a20383c5cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:14.995822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b9633-1817-4570-8f54-d8a20383c5cb service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:15.005831 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 982/1958] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:15.414379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a9a2e-c34b-4f17-9dd7-783f9cad3d7d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:15.417154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-951a9a2e-c34b-4f17-9dd7-783f9cad3d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:15.450609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a9a2e-c34b-4f17-9dd7-783f9cad3d7d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:15.467617 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:15.471420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.472220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.472605 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] PATH_INFO: `/v3/projects/c902cd4854314a84b311d5a5c8b0927d` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.477255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c902cd4854314a84b311d5a5c8b0927d)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:15.479422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:15.589612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c902cd4854314a84b311d5a5c8b0927d', 'request_id': 'req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.601166 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c902cd4854314a84b311d5a5c8b0927d', 'request_id': 'req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.608876 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c902cd4854314a84b311d5a5c8b0927d', 'request_id': 'req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.632159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] The token cache is being invalidate because project c902cd4854314a84b311d5a5c8b0927d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:15.632461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.633245 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.636154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 977/1959] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:15 2022] DELETE /identity/v3/projects/c902cd4854314a84b311d5a5c8b0927d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:15.641353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b9633-1817-4570-8f54-d8a20383c5cb service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:15.652896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b9633-1817-4570-8f54-d8a20383c5cb service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:15.715483 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d7952-c0df-4b41-818b-d133a3d59c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:15.716372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d7952-c0df-4b41-818b-d133a3d59c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.716661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d7952-c0df-4b41-818b-d133a3d59c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.716946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d7952-c0df-4b41-818b-d133a3d59c7f admin admin] PATH_INFO: `/v3/users/2ed4fcaed5804cf7917a122cf8234a52` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.718142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d7952-c0df-4b41-818b-d133a3d59c7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed4fcaed5804cf7917a122cf8234a52)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:15.720142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d7952-c0df-4b41-818b-d133a3d59c7f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:15.755763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:15.769813 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a71d7952-c0df-4b41-818b-d133a3d59c7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed4fcaed5804cf7917a122cf8234a52'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.776990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a71d7952-c0df-4b41-818b-d133a3d59c7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed4fcaed5804cf7917a122cf8234a52'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.777613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a71d7952-c0df-4b41-818b-d133a3d59c7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed4fcaed5804cf7917a122cf8234a52'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.784879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a71d7952-c0df-4b41-818b-d133a3d59c7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed4fcaed5804cf7917a122cf8234a52'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.785679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a71d7952-c0df-4b41-818b-d133a3d59c7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed4fcaed5804cf7917a122cf8234a52'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.807864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 983/1960] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:15 2022] DELETE /identity/v3/users/2ed4fcaed5804cf7917a122cf8234a52 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:16.098319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.099796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:16.114882 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4040e-6dcf-47f1-bc0e-fe1ffb95b30c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.186708 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553332a-ee30-4c0c-bcb5-2ec24220e125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.187613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553332a-ee30-4c0c-bcb5-2ec24220e125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.187907 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553332a-ee30-4c0c-bcb5-2ec24220e125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.188220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553332a-ee30-4c0c-bcb5-2ec24220e125 admin admin] PATH_INFO: `/v3/projects/d31c6625e6d046b1b4fdd19fd8b6dc2b` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.189547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553332a-ee30-4c0c-bcb5-2ec24220e125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d31c6625e6d046b1b4fdd19fd8b6dc2b)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.191796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553332a-ee30-4c0c-bcb5-2ec24220e125 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.243466 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2553332a-ee30-4c0c-bcb5-2ec24220e125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd31c6625e6d046b1b4fdd19fd8b6dc2b', 'request_id': 'req-2553332a-ee30-4c0c-bcb5-2ec24220e125'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:16.249867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2553332a-ee30-4c0c-bcb5-2ec24220e125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd31c6625e6d046b1b4fdd19fd8b6dc2b', 'request_id': 'req-2553332a-ee30-4c0c-bcb5-2ec24220e125'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:16.255602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2553332a-ee30-4c0c-bcb5-2ec24220e125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd31c6625e6d046b1b4fdd19fd8b6dc2b', 'request_id': 'req-2553332a-ee30-4c0c-bcb5-2ec24220e125'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:16.268962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2553332a-ee30-4c0c-bcb5-2ec24220e125 admin admin] The token cache is being invalidate because project d31c6625e6d046b1b4fdd19fd8b6dc2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:16.269256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2553332a-ee30-4c0c-bcb5-2ec24220e125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:16.269982 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2553332a-ee30-4c0c-bcb5-2ec24220e125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:16.271646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 978/1961] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:16 2022] DELETE /identity/v3/projects/d31c6625e6d046b1b4fdd19fd8b6dc2b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:16.276985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12a698-b6d8-4321-a015-fb8621191364 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.277368 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12a698-b6d8-4321-a015-fb8621191364 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.277650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12a698-b6d8-4321-a015-fb8621191364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.298942 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-1b12a698-b6d8-4321-a015-fb8621191364 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:16.300269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1b12a698-b6d8-4321-a015-fb8621191364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:16.369292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 984/1962] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:16.377692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586c7c0-24b1-4e15-ac82-0303e3b2c127 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.378012 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586c7c0-24b1-4e15-ac82-0303e3b2c127 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.378297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586c7c0-24b1-4e15-ac82-0303e3b2c127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.401272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1586c7c0-24b1-4e15-ac82-0303e3b2c127 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:16.402085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1586c7c0-24b1-4e15-ac82-0303e3b2c127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:16.411107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 979/1963] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:16.416661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12a698-b6d8-4321-a015-fb8621191364 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.429813 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12a698-b6d8-4321-a015-fb8621191364 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.436940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9685cda-eba6-416f-8657-2f87ae58e45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.437705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9685cda-eba6-416f-8657-2f87ae58e45c admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.437996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9685cda-eba6-416f-8657-2f87ae58e45c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.438311 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9685cda-eba6-416f-8657-2f87ae58e45c admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.439032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9685cda-eba6-416f-8657-2f87ae58e45c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:16.439760 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9685cda-eba6-416f-8657-2f87ae58e45c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.441726 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9685cda-eba6-416f-8657-2f87ae58e45c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.463570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 985/1964] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:08:16.465175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586c7c0-24b1-4e15-ac82-0303e3b2c127 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.470354 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:16.476385 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586c7c0-24b1-4e15-ac82-0303e3b2c127 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.483961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2e015-c5dc-4f6d-81ec-83e073aa0da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.484776 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2e015-c5dc-4f6d-81ec-83e073aa0da5 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.485128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2e015-c5dc-4f6d-81ec-83e073aa0da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.485421 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2e015-c5dc-4f6d-81ec-83e073aa0da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.486813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2e015-c5dc-4f6d-81ec-83e073aa0da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.489137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2e015-c5dc-4f6d-81ec-83e073aa0da5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.522538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 980/1965] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:16 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:16.527092 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9685cda-eba6-416f-8657-2f87ae58e45c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.537145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9685cda-eba6-416f-8657-2f87ae58e45c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.608240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777b0b4-e10c-413b-8744-12860a85688c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.609031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777b0b4-e10c-413b-8744-12860a85688c admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.609448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777b0b4-e10c-413b-8744-12860a85688c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.609830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777b0b4-e10c-413b-8744-12860a85688c admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.610625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777b0b4-e10c-413b-8744-12860a85688c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.612681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777b0b4-e10c-413b-8744-12860a85688c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.636012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 986/1966] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:16 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:16.639984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2e015-c5dc-4f6d-81ec-83e073aa0da5 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.651585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2e015-c5dc-4f6d-81ec-83e073aa0da5 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.656192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b25c22-e38c-4255-b051-dd0af15ccebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.657083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b25c22-e38c-4255-b051-dd0af15ccebc admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.657283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b25c22-e38c-4255-b051-dd0af15ccebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.657527 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b25c22-e38c-4255-b051-dd0af15ccebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.658267 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b25c22-e38c-4255-b051-dd0af15ccebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.660053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b25c22-e38c-4255-b051-dd0af15ccebc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.673181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 981/1967] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:16.677004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777b0b4-e10c-413b-8744-12860a85688c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.692801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777b0b4-e10c-413b-8744-12860a85688c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.706181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd956c-f0e1-4538-92dd-b23b137dcc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.707026 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd956c-f0e1-4538-92dd-b23b137dcc5f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.707491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd956c-f0e1-4538-92dd-b23b137dcc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.707827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd956c-f0e1-4538-92dd-b23b137dcc5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.708620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd956c-f0e1-4538-92dd-b23b137dcc5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.710547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd956c-f0e1-4538-92dd-b23b137dcc5f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.722978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 987/1968] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:16.727015 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b25c22-e38c-4255-b051-dd0af15ccebc admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.737970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b25c22-e38c-4255-b051-dd0af15ccebc admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.749379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906cd1bc-3d58-4eb5-b942-e8b990adf3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.750357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cd1bc-3d58-4eb5-b942-e8b990adf3db admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.750649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cd1bc-3d58-4eb5-b942-e8b990adf3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.750930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cd1bc-3d58-4eb5-b942-e8b990adf3db admin admin] PATH_INFO: `/v3/projects/4d540418e8c9430ebda6cffab621fde1/users/dce339516d00430a8d56465f51f8a2dd/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.771188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906cd1bc-3d58-4eb5-b942-e8b990adf3db admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d540418e8c9430ebda6cffab621fde1, user_id=dce339516d00430a8d56465f51f8a2dd, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.773957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906cd1bc-3d58-4eb5-b942-e8b990adf3db admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.788625 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 982/1969] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:16 2022] PUT /identity/v3/projects/4d540418e8c9430ebda6cffab621fde1/users/dce339516d00430a8d56465f51f8a2dd/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:16.799310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8773-3a39-4636-aaff-31deca6b05a6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.799767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8773-3a39-4636-aaff-31deca6b05a6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.800236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8773-3a39-4636-aaff-31deca6b05a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.829135 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f98d8773-3a39-4636-aaff-31deca6b05a6 None None] MFA Rules not processed for user `dce339516d00430a8d56465f51f8a2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:16.830434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f98d8773-3a39-4636-aaff-31deca6b05a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:16.834578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:16.904138 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 988/1970] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:16.916336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906cd1bc-3d58-4eb5-b942-e8b990adf3db admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.918054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-906cd1bc-3d58-4eb5-b942-e8b990adf3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:16.976708 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906cd1bc-3d58-4eb5-b942-e8b990adf3db admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.980403 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d43b9-0eef-480b-8af5-f2231ead306d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.981059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d43b9-0eef-480b-8af5-f2231ead306d service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.981347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d43b9-0eef-480b-8af5-f2231ead306d service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.981629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d43b9-0eef-480b-8af5-f2231ead306d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.990129 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d43b9-0eef-480b-8af5-f2231ead306d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.991648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d43b9-0eef-480b-8af5-f2231ead306d service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:17.000994 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 983/1971] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:17.020195 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8773-3a39-4636-aaff-31deca6b05a6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:17.086751 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8773-3a39-4636-aaff-31deca6b05a6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:17.091432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:17.092319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:17.092607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:17.092892 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] PATH_INFO: `/v3/users/dce339516d00430a8d56465f51f8a2dd` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:17.094119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dce339516d00430a8d56465f51f8a2dd)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:17.096109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:17.148309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dce339516d00430a8d56465f51f8a2dd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:17.156005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dce339516d00430a8d56465f51f8a2dd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:17.156654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dce339516d00430a8d56465f51f8a2dd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:17.164412 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dce339516d00430a8d56465f51f8a2dd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:17.165148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dce339516d00430a8d56465f51f8a2dd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:17.183416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 989/1972] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:17 2022] DELETE /identity/v3/users/dce339516d00430a8d56465f51f8a2dd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:17.187788 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc690f-6c78-4b65-b1d5-5009aebc50bd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:17.188103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc690f-6c78-4b65-b1d5-5009aebc50bd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:17.188377 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc690f-6c78-4b65-b1d5-5009aebc50bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:17.207216 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e6dc690f-6c78-4b65-b1d5-5009aebc50bd None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:17.208467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e6dc690f-6c78-4b65-b1d5-5009aebc50bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:17.271168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 984/1973] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:17.280612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:17.282010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:17.321070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:17.357741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560c3f9-b2d7-49b4-92c1-fd99e006261f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:17.361401 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c118692-69eb-49be-b238-c16cc1ce411f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:17.362054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c118692-69eb-49be-b238-c16cc1ce411f service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:17.362343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c118692-69eb-49be-b238-c16cc1ce411f service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:17.362619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c118692-69eb-49be-b238-c16cc1ce411f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:17.373413 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c118692-69eb-49be-b238-c16cc1ce411f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:17.375581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c118692-69eb-49be-b238-c16cc1ce411f service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:17.387071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 990/1974] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:17.938433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc690f-6c78-4b65-b1d5-5009aebc50bd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:17.944852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e6dc690f-6c78-4b65-b1d5-5009aebc50bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:17.968342 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc690f-6c78-4b65-b1d5-5009aebc50bd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:17.977445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af82ab-535d-43ca-823f-94abc1908efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:17.979186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af82ab-535d-43ca-823f-94abc1908efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:17.979640 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af82ab-535d-43ca-823f-94abc1908efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:17.979977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af82ab-535d-43ca-823f-94abc1908efb admin admin] PATH_INFO: `/v3/projects/4d540418e8c9430ebda6cffab621fde1` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:17.981255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af82ab-535d-43ca-823f-94abc1908efb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d540418e8c9430ebda6cffab621fde1)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:17.983419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af82ab-535d-43ca-823f-94abc1908efb admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:18.031726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:18.035980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-68af82ab-535d-43ca-823f-94abc1908efb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d540418e8c9430ebda6cffab621fde1', 'request_id': 'req-68af82ab-535d-43ca-823f-94abc1908efb'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:18.044823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-68af82ab-535d-43ca-823f-94abc1908efb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d540418e8c9430ebda6cffab621fde1', 'request_id': 'req-68af82ab-535d-43ca-823f-94abc1908efb'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:18.051796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-68af82ab-535d-43ca-823f-94abc1908efb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d540418e8c9430ebda6cffab621fde1', 'request_id': 'req-68af82ab-535d-43ca-823f-94abc1908efb'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:18.065672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-68af82ab-535d-43ca-823f-94abc1908efb admin admin] The token cache is being invalidate because project 4d540418e8c9430ebda6cffab621fde1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:18.065969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-68af82ab-535d-43ca-823f-94abc1908efb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:18.066600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-68af82ab-535d-43ca-823f-94abc1908efb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:18.068260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 985/1975] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:17 2022] DELETE /identity/v3/projects/4d540418e8c9430ebda6cffab621fde1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:18.073770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede62ad-e77d-4023-b8ac-e539090d50b6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:18.074237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede62ad-e77d-4023-b8ac-e539090d50b6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:18.074312 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede62ad-e77d-4023-b8ac-e539090d50b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:18.093852 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0ede62ad-e77d-4023-b8ac-e539090d50b6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:18.095055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0ede62ad-e77d-4023-b8ac-e539090d50b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:18.156401 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 991/1976] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:18.165027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b203c3-f4ca-48a7-9b94-98ce32e17e4e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:18.165327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b203c3-f4ca-48a7-9b94-98ce32e17e4e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:18.165640 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b203c3-f4ca-48a7-9b94-98ce32e17e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:18.185529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-81b203c3-f4ca-48a7-9b94-98ce32e17e4e None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:18.186783 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-81b203c3-f4ca-48a7-9b94-98ce32e17e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:18.195544 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 986/1977] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:18.200383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede62ad-e77d-4023-b8ac-e539090d50b6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:18.210307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede62ad-e77d-4023-b8ac-e539090d50b6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:18.215715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fa9f3-bede-4d58-a81f-8aac8930b4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:18.216373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fa9f3-bede-4d58-a81f-8aac8930b4ff admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:18.216660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fa9f3-bede-4d58-a81f-8aac8930b4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:18.216937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fa9f3-bede-4d58-a81f-8aac8930b4ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:18.217659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fa9f3-bede-4d58-a81f-8aac8930b4ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:18.217966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fa9f3-bede-4d58-a81f-8aac8930b4ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:18.219594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fa9f3-bede-4d58-a81f-8aac8930b4ff admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:18.231543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 992/1978] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:08:18.235588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b203c3-f4ca-48a7-9b94-98ce32e17e4e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:18.245133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b203c3-f4ca-48a7-9b94-98ce32e17e4e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:18.249895 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0851122-847a-472b-982a-7ff1276a904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:18.250556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0851122-847a-472b-982a-7ff1276a904b admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:18.250847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0851122-847a-472b-982a-7ff1276a904b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:18.251131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0851122-847a-472b-982a-7ff1276a904b admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:18.251871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0851122-847a-472b-982a-7ff1276a904b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:18.253797 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0851122-847a-472b-982a-7ff1276a904b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:18.285245 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 987/1979] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:18 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:18.289532 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fa9f3-bede-4d58-a81f-8aac8930b4ff admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:18.302117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fa9f3-bede-4d58-a81f-8aac8930b4ff admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:18.362684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42989c-b0a6-4e94-8742-d5518bb88ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:18.363401 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42989c-b0a6-4e94-8742-d5518bb88ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:18.363688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42989c-b0a6-4e94-8742-d5518bb88ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:18.363973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42989c-b0a6-4e94-8742-d5518bb88ae5 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:18.364680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42989c-b0a6-4e94-8742-d5518bb88ae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:18.366632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42989c-b0a6-4e94-8742-d5518bb88ae5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:18.387334 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 993/1980] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:18 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:18.391468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0851122-847a-472b-982a-7ff1276a904b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:18.402497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0851122-847a-472b-982a-7ff1276a904b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:18.407139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdf598-6fb9-47b5-8022-471263d3d4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:18.407842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdf598-6fb9-47b5-8022-471263d3d4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:18.408133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdf598-6fb9-47b5-8022-471263d3d4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:18.408414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdf598-6fb9-47b5-8022-471263d3d4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:18.409144 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdf598-6fb9-47b5-8022-471263d3d4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:18.410746 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdf598-6fb9-47b5-8022-471263d3d4d5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:18.425676 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 988/1981] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:18.430045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42989c-b0a6-4e94-8742-d5518bb88ae5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:18.440047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42989c-b0a6-4e94-8742-d5518bb88ae5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:18.444864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5a369-0a5c-4d16-82af-ff5ea0e8af8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:18.445531 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5a369-0a5c-4d16-82af-ff5ea0e8af8a admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:18.445818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5a369-0a5c-4d16-82af-ff5ea0e8af8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:18.446101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5a369-0a5c-4d16-82af-ff5ea0e8af8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:18.446877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5a369-0a5c-4d16-82af-ff5ea0e8af8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:18.448527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5a369-0a5c-4d16-82af-ff5ea0e8af8a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:18.462274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 994/1982] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:18.464174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdf598-6fb9-47b5-8022-471263d3d4d5 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:18.474782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdf598-6fb9-47b5-8022-471263d3d4d5 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:18.480180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9b825-4277-4d3a-9d6b-9c62b336fdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:18.481179 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9b825-4277-4d3a-9d6b-9c62b336fdad admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:18.481526 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9b825-4277-4d3a-9d6b-9c62b336fdad admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:18.481871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9b825-4277-4d3a-9d6b-9c62b336fdad admin admin] PATH_INFO: `/v3/projects/8bf3d02b9d8d431b9b1a76dba93f77e2/users/94144004dab94e7aa60f33c308903b00/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:18.502241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9b825-4277-4d3a-9d6b-9c62b336fdad admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf3d02b9d8d431b9b1a76dba93f77e2, user_id=94144004dab94e7aa60f33c308903b00, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:18.504892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9b825-4277-4d3a-9d6b-9c62b336fdad admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:18.519724 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 989/1983] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:18 2022] PUT /identity/v3/projects/8bf3d02b9d8d431b9b1a76dba93f77e2/users/94144004dab94e7aa60f33c308903b00/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:18.532696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb122e5-9c9f-4b90-b62a-7272e0f605a2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:18.532988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb122e5-9c9f-4b90-b62a-7272e0f605a2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:18.533272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb122e5-9c9f-4b90-b62a-7272e0f605a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:18.560189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2cb122e5-9c9f-4b90-b62a-7272e0f605a2 None None] MFA Rules not processed for user `94144004dab94e7aa60f33c308903b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:18.561393 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2cb122e5-9c9f-4b90-b62a-7272e0f605a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:18.581594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:18.637226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 995/1984] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:18.648747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9b825-4277-4d3a-9d6b-9c62b336fdad admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:18.651333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-afb9b825-4277-4d3a-9d6b-9c62b336fdad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:18.709139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9b825-4277-4d3a-9d6b-9c62b336fdad admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:18.712737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5b6ae-96c3-49c8-b57b-27c3f0925af8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:18.713887 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5b6ae-96c3-49c8-b57b-27c3f0925af8 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:18.714187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5b6ae-96c3-49c8-b57b-27c3f0925af8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:18.714451 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5b6ae-96c3-49c8-b57b-27c3f0925af8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:18.722531 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5b6ae-96c3-49c8-b57b-27c3f0925af8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:18.723990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5b6ae-96c3-49c8-b57b-27c3f0925af8 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:18.733376 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 990/1985] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:23.747924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb122e5-9c9f-4b90-b62a-7272e0f605a2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:23.749632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2cb122e5-9c9f-4b90-b62a-7272e0f605a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:23.851691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb122e5-9c9f-4b90-b62a-7272e0f605a2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:23.858833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:23.858833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:23.859410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:23.860013 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] PATH_INFO: `/v3/users/f3b81904965341b19b7221220a55cee3` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:23.861483 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b81904965341b19b7221220a55cee3)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:23.863718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:23.938983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b81904965341b19b7221220a55cee3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:23.948767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b81904965341b19b7221220a55cee3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:23.949543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b81904965341b19b7221220a55cee3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:23.957349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b81904965341b19b7221220a55cee3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:23.958053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b81904965341b19b7221220a55cee3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:23.977078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 996/1986] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:23 2022] DELETE /identity/v3/users/f3b81904965341b19b7221220a55cee3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:23.981859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370271f4-b08a-4eb9-a351-53ac3764a52f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:23.982442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370271f4-b08a-4eb9-a351-53ac3764a52f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:23.982576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370271f4-b08a-4eb9-a351-53ac3764a52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:24.001647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-370271f4-b08a-4eb9-a351-53ac3764a52f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:24.002861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-370271f4-b08a-4eb9-a351-53ac3764a52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:24.070211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 991/1987] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:24.079833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:24.081325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:24.138080 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:24.151874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4a60c-c7ab-4530-bd09-8e18a0406290 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:24.155504 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828ca3c-0587-452e-9046-34238fcd0906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:24.156155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828ca3c-0587-452e-9046-34238fcd0906 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:24.156436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828ca3c-0587-452e-9046-34238fcd0906 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:24.156716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828ca3c-0587-452e-9046-34238fcd0906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:24.167266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828ca3c-0587-452e-9046-34238fcd0906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:24.168887 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828ca3c-0587-452e-9046-34238fcd0906 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:24.180099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 997/1988] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:24.475694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370271f4-b08a-4eb9-a351-53ac3764a52f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:24.477226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-370271f4-b08a-4eb9-a351-53ac3764a52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:24.490240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370271f4-b08a-4eb9-a351-53ac3764a52f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:24.495684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:24.496619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:24.496976 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:24.497271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] PATH_INFO: `/v3/projects/3e7500e9c69444c2a10857432d92688f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:24.498469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e7500e9c69444c2a10857432d92688f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:24.500462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:24.507563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:24.553589 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e7500e9c69444c2a10857432d92688f', 'request_id': 'req-3e161d66-ceba-448e-8567-8bfcf991ee03'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:24.560438 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e7500e9c69444c2a10857432d92688f', 'request_id': 'req-3e161d66-ceba-448e-8567-8bfcf991ee03'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:24.565999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e7500e9c69444c2a10857432d92688f', 'request_id': 'req-3e161d66-ceba-448e-8567-8bfcf991ee03'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:24.578699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] The token cache is being invalidate because project 3e7500e9c69444c2a10857432d92688f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:24.578989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:24.579628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:24.581149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 992/1989] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:24 2022] DELETE /identity/v3/projects/3e7500e9c69444c2a10857432d92688f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:24.585267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828ca3c-0587-452e-9046-34238fcd0906 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:24.594884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828ca3c-0587-452e-9046-34238fcd0906 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:24.652964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868625ae-b8fe-42d7-ba62-7f825f16e7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:24.653851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868625ae-b8fe-42d7-ba62-7f825f16e7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:24.654165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868625ae-b8fe-42d7-ba62-7f825f16e7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:24.654445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868625ae-b8fe-42d7-ba62-7f825f16e7db admin admin] PATH_INFO: `/v3/users/584f515aeb2d419c8e8cabb406269297` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:24.655701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868625ae-b8fe-42d7-ba62-7f825f16e7db admin admin] RBAC: Authorizing `identity:delete_user(user_id=584f515aeb2d419c8e8cabb406269297)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:24.657655 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868625ae-b8fe-42d7-ba62-7f825f16e7db admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:24.706398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-868625ae-b8fe-42d7-ba62-7f825f16e7db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '584f515aeb2d419c8e8cabb406269297'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:24.713941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-868625ae-b8fe-42d7-ba62-7f825f16e7db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '584f515aeb2d419c8e8cabb406269297'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:24.714654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-868625ae-b8fe-42d7-ba62-7f825f16e7db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '584f515aeb2d419c8e8cabb406269297'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:24.721590 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-868625ae-b8fe-42d7-ba62-7f825f16e7db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '584f515aeb2d419c8e8cabb406269297'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:24.722238 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-868625ae-b8fe-42d7-ba62-7f825f16e7db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '584f515aeb2d419c8e8cabb406269297'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:24.740256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 998/1990] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:24 2022] DELETE /identity/v3/users/584f515aeb2d419c8e8cabb406269297 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:25.108737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:25.110686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:25.152110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e161d66-ceba-448e-8567-8bfcf991ee03 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:25.248182 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a2514-a33b-4078-a763-a97da55a90bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:25.249199 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a2514-a33b-4078-a763-a97da55a90bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:25.249563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a2514-a33b-4078-a763-a97da55a90bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:25.249914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a2514-a33b-4078-a763-a97da55a90bb admin admin] PATH_INFO: `/v3/projects/b41603209cd140d886232bc729a17e41` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:25.251186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a2514-a33b-4078-a763-a97da55a90bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b41603209cd140d886232bc729a17e41)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:25.253240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a2514-a33b-4078-a763-a97da55a90bb admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:25.335454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-936a2514-a33b-4078-a763-a97da55a90bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b41603209cd140d886232bc729a17e41', 'request_id': 'req-936a2514-a33b-4078-a763-a97da55a90bb'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:25.341884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-936a2514-a33b-4078-a763-a97da55a90bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b41603209cd140d886232bc729a17e41', 'request_id': 'req-936a2514-a33b-4078-a763-a97da55a90bb'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:25.348134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-936a2514-a33b-4078-a763-a97da55a90bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b41603209cd140d886232bc729a17e41', 'request_id': 'req-936a2514-a33b-4078-a763-a97da55a90bb'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:25.363293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-936a2514-a33b-4078-a763-a97da55a90bb admin admin] The token cache is being invalidate because project b41603209cd140d886232bc729a17e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:25.363556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-936a2514-a33b-4078-a763-a97da55a90bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:25.364216 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-936a2514-a33b-4078-a763-a97da55a90bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:25.366175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 993/1991] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:25 2022] DELETE /identity/v3/projects/b41603209cd140d886232bc729a17e41 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:25.371515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facbf52-28de-4fbb-80cc-a323e643789f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:25.372063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facbf52-28de-4fbb-80cc-a323e643789f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:25.372165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facbf52-28de-4fbb-80cc-a323e643789f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:25.396026 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-3facbf52-28de-4fbb-80cc-a323e643789f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:25.398193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3facbf52-28de-4fbb-80cc-a323e643789f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:25.470029 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 999/1992] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:25.477995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718231a2-c291-4a2b-b473-12249ec3f160 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:25.478276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718231a2-c291-4a2b-b473-12249ec3f160 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:25.478601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718231a2-c291-4a2b-b473-12249ec3f160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:25.497850 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-718231a2-c291-4a2b-b473-12249ec3f160 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:25.499088 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-718231a2-c291-4a2b-b473-12249ec3f160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:25.507477 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 994/1993] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:25.512464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facbf52-28de-4fbb-80cc-a323e643789f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:25.518672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:25.522636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facbf52-28de-4fbb-80cc-a323e643789f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:25.528978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675e1c7-290f-4a53-ad5f-d2a043b45637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:25.529973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675e1c7-290f-4a53-ad5f-d2a043b45637 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:25.530559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675e1c7-290f-4a53-ad5f-d2a043b45637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:25.530806 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675e1c7-290f-4a53-ad5f-d2a043b45637 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:25.531761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675e1c7-290f-4a53-ad5f-d2a043b45637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:25.532081 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675e1c7-290f-4a53-ad5f-d2a043b45637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:25.533696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675e1c7-290f-4a53-ad5f-d2a043b45637 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:25.546548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1000/1994] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:08:25.550931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718231a2-c291-4a2b-b473-12249ec3f160 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:25.561101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718231a2-c291-4a2b-b473-12249ec3f160 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:25.565785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5659f-633d-47ef-97f1-7c7485b7ea54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:25.566451 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5659f-633d-47ef-97f1-7c7485b7ea54 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:25.566732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5659f-633d-47ef-97f1-7c7485b7ea54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:25.567007 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5659f-633d-47ef-97f1-7c7485b7ea54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:25.567746 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5659f-633d-47ef-97f1-7c7485b7ea54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:25.569643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5659f-633d-47ef-97f1-7c7485b7ea54 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:25.583896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:25.597809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 995/1995] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:25 2022] POST /identity/v3/projects => generated 409 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:25.601985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675e1c7-290f-4a53-ad5f-d2a043b45637 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:25.612621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675e1c7-290f-4a53-ad5f-d2a043b45637 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:25.670534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c87785-7ac8-48e7-99b9-4c58d6d746c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:25.671948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c87785-7ac8-48e7-99b9-4c58d6d746c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:25.671948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c87785-7ac8-48e7-99b9-4c58d6d746c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:25.671948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c87785-7ac8-48e7-99b9-4c58d6d746c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:25.672632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c87785-7ac8-48e7-99b9-4c58d6d746c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:25.674456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c87785-7ac8-48e7-99b9-4c58d6d746c0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:25.697100 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1001/1996] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:25 2022] POST /identity/v3/users => generated 305 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:25.703523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5659f-633d-47ef-97f1-7c7485b7ea54 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:25.714264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5659f-633d-47ef-97f1-7c7485b7ea54 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:25.718954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af3cbe-4dcc-4f8f-a16e-cfa335e5e05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:25.719662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af3cbe-4dcc-4f8f-a16e-cfa335e5e05e admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:25.719955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af3cbe-4dcc-4f8f-a16e-cfa335e5e05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:25.720238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af3cbe-4dcc-4f8f-a16e-cfa335e5e05e admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:25.720973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af3cbe-4dcc-4f8f-a16e-cfa335e5e05e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:25.722579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af3cbe-4dcc-4f8f-a16e-cfa335e5e05e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:25.733744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 996/1997] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:25.738762 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c87785-7ac8-48e7-99b9-4c58d6d746c0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:25.749432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c87785-7ac8-48e7-99b9-4c58d6d746c0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:25.754056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129f13f-d791-45b7-80f2-54a5b4e10b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:25.754709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129f13f-d791-45b7-80f2-54a5b4e10b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:25.754998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129f13f-d791-45b7-80f2-54a5b4e10b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:25.755279 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129f13f-d791-45b7-80f2-54a5b4e10b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:25.756040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129f13f-d791-45b7-80f2-54a5b4e10b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:25.757746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129f13f-d791-45b7-80f2-54a5b4e10b09 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:25.767746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1002/1998] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:25.772577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af3cbe-4dcc-4f8f-a16e-cfa335e5e05e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:25.785449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af3cbe-4dcc-4f8f-a16e-cfa335e5e05e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:25.792081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500c82f-72af-40fb-8f9a-cc1556660bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:25.793218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500c82f-72af-40fb-8f9a-cc1556660bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:25.793828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500c82f-72af-40fb-8f9a-cc1556660bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:25.793958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500c82f-72af-40fb-8f9a-cc1556660bcf admin admin] PATH_INFO: `/v3/projects/832843c38216472dae48b1f446e8a2f9/users/604eb9039ce7400da44529f2c3905e61/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:25.814491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500c82f-72af-40fb-8f9a-cc1556660bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=832843c38216472dae48b1f446e8a2f9, user_id=604eb9039ce7400da44529f2c3905e61, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:25.818280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500c82f-72af-40fb-8f9a-cc1556660bcf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:25.834030 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 997/1999] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:25 2022] PUT /identity/v3/projects/832843c38216472dae48b1f446e8a2f9/users/604eb9039ce7400da44529f2c3905e61/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:25.852107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a65ee-ff3a-4e1f-9c2b-8dd6ce5814ae None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:25.852528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a65ee-ff3a-4e1f-9c2b-8dd6ce5814ae None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:25.852870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a65ee-ff3a-4e1f-9c2b-8dd6ce5814ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:25.887731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-571a65ee-ff3a-4e1f-9c2b-8dd6ce5814ae None None] MFA Rules not processed for user `604eb9039ce7400da44529f2c3905e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:25.888950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-571a65ee-ff3a-4e1f-9c2b-8dd6ce5814ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:25.964812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1003/2000] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:25 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:25.969171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500c82f-72af-40fb-8f9a-cc1556660bcf admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:26.042644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500c82f-72af-40fb-8f9a-cc1556660bcf admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:26.047886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e5875-b7f4-473c-a187-2d3c0ef290cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:26.048612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e5875-b7f4-473c-a187-2d3c0ef290cf admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:26.048978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e5875-b7f4-473c-a187-2d3c0ef290cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:26.049288 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e5875-b7f4-473c-a187-2d3c0ef290cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:26.050037 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e5875-b7f4-473c-a187-2d3c0ef290cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:26.052237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e5875-b7f4-473c-a187-2d3c0ef290cf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:26.084552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 998/2001] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:25 2022] POST /identity/v3/projects => generated 409 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:26.089400 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a65ee-ff3a-4e1f-9c2b-8dd6ce5814ae None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:26.101475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a65ee-ff3a-4e1f-9c2b-8dd6ce5814ae None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:26.168015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acf4b0-02b7-4576-997c-c57a2c338637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:26.168801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acf4b0-02b7-4576-997c-c57a2c338637 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:26.169154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acf4b0-02b7-4576-997c-c57a2c338637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:26.169503 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acf4b0-02b7-4576-997c-c57a2c338637 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:26.170283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acf4b0-02b7-4576-997c-c57a2c338637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:26.172371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acf4b0-02b7-4576-997c-c57a2c338637 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:26.179858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:26.218765 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1004/2002] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:26 2022] POST /identity/v3/users => generated 311 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:26.219833 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e5875-b7f4-473c-a187-2d3c0ef290cf admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:26.237059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e5875-b7f4-473c-a187-2d3c0ef290cf admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:26.247153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5fa5a-c769-412f-81f0-8cdd711ae347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:26.247956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5fa5a-c769-412f-81f0-8cdd711ae347 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:26.248309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5fa5a-c769-412f-81f0-8cdd711ae347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:26.248681 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5fa5a-c769-412f-81f0-8cdd711ae347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:26.249493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5fa5a-c769-412f-81f0-8cdd711ae347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:26.251479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5fa5a-c769-412f-81f0-8cdd711ae347 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:26.264286 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 999/2003] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:08:26.275495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acf4b0-02b7-4576-997c-c57a2c338637 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:26.281849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acf4b0-02b7-4576-997c-c57a2c338637 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:26.288952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5d1c3-395e-4ed9-86c0-3cd7558192c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:26.290572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5d1c3-395e-4ed9-86c0-3cd7558192c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:26.291727 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5d1c3-395e-4ed9-86c0-3cd7558192c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:26.292245 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5d1c3-395e-4ed9-86c0-3cd7558192c7 admin admin] PATH_INFO: `/v3/projects/069d02cc804a42c987d006b779ddcb3b/users/b2c2304cac3242149be6132a4b101ada/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:26.320746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5d1c3-395e-4ed9-86c0-3cd7558192c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=069d02cc804a42c987d006b779ddcb3b, user_id=b2c2304cac3242149be6132a4b101ada, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:26.325593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5d1c3-395e-4ed9-86c0-3cd7558192c7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:26.348644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1005/2004] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:26 2022] PUT /identity/v3/projects/069d02cc804a42c987d006b779ddcb3b/users/b2c2304cac3242149be6132a4b101ada/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:26.352385 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5fa5a-c769-412f-81f0-8cdd711ae347 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:26.366634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5fa5a-c769-412f-81f0-8cdd711ae347 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:26.453960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9b9f1-af7f-4461-8dcc-7bf400229984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:26.454635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9b9f1-af7f-4461-8dcc-7bf400229984 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:26.455063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9b9f1-af7f-4461-8dcc-7bf400229984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:26.455563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9b9f1-af7f-4461-8dcc-7bf400229984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:26.456389 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9b9f1-af7f-4461-8dcc-7bf400229984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:26.458267 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9b9f1-af7f-4461-8dcc-7bf400229984 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:26.473380 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1000/2005] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:26.478016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5d1c3-395e-4ed9-86c0-3cd7558192c7 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:26.489837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5d1c3-395e-4ed9-86c0-3cd7558192c7 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:26.495597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ec03-fbb6-49b9-bae9-ffe6412c5ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:26.496500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ec03-fbb6-49b9-bae9-ffe6412c5ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:26.496780 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ec03-fbb6-49b9-bae9-ffe6412c5ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:26.497055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ec03-fbb6-49b9-bae9-ffe6412c5ccb admin admin] PATH_INFO: `/v3/projects/069d02cc804a42c987d006b779ddcb3b/users/b2c2304cac3242149be6132a4b101ada/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:26.501128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854ec03-fbb6-49b9-bae9-ffe6412c5ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=069d02cc804a42c987d006b779ddcb3b, user_id=b2c2304cac3242149be6132a4b101ada, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:26.503999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854ec03-fbb6-49b9-bae9-ffe6412c5ccb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:26.516452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1006/2006] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:26 2022] PUT /identity/v3/projects/069d02cc804a42c987d006b779ddcb3b/users/b2c2304cac3242149be6132a4b101ada/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:26.527303 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fe663-108b-4ccb-ab10-2199c226afb5 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:26.527680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fe663-108b-4ccb-ab10-2199c226afb5 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:26.527962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fe663-108b-4ccb-ab10-2199c226afb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:26.555812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e23fe663-108b-4ccb-ab10-2199c226afb5 None None] MFA Rules not processed for user `b2c2304cac3242149be6132a4b101ada`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:26.557612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e23fe663-108b-4ccb-ab10-2199c226afb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:26.575935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:26.628136 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1001/2007] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:26 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:26.638585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ec03-fbb6-49b9-bae9-ffe6412c5ccb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:26.640323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d854ec03-fbb6-49b9-bae9-ffe6412c5ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:26.697332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ec03-fbb6-49b9-bae9-ffe6412c5ccb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:26.700971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf7c81-e874-462b-b2bf-c07b1dade241 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:26.701615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf7c81-e874-462b-b2bf-c07b1dade241 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:26.701898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf7c81-e874-462b-b2bf-c07b1dade241 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:26.702174 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf7c81-e874-462b-b2bf-c07b1dade241 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:26.711009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf7c81-e874-462b-b2bf-c07b1dade241 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:26.712468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf7c81-e874-462b-b2bf-c07b1dade241 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:26.722258 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1007/2008] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:26 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:26.875656 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fe663-108b-4ccb-ab10-2199c226afb5 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:26.885113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fe663-108b-4ccb-ab10-2199c226afb5 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:26.888650 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186a494-a1b1-4952-a14b-d45b15de5797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:26.889289 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186a494-a1b1-4952-a14b-d45b15de5797 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:26.889573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186a494-a1b1-4952-a14b-d45b15de5797 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:26.889846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186a494-a1b1-4952-a14b-d45b15de5797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:26.947871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186a494-a1b1-4952-a14b-d45b15de5797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:26.949333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186a494-a1b1-4952-a14b-d45b15de5797 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:26.958779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1002/2009] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:26 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:36.718162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=5,keystone:SELECT=43 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:36.954659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=48 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:44.940141 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf7c81-e874-462b-b2bf-c07b1dade241 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:45.032368 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf7c81-e874-462b-b2bf-c07b1dade241 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:45.037355 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:45.038477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:45.038842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:45.040428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] PATH_INFO: `/v3/users/604eb9039ce7400da44529f2c3905e61` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:45.041791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=604eb9039ce7400da44529f2c3905e61)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:45.043820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:45.160076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '604eb9039ce7400da44529f2c3905e61'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.170425 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '604eb9039ce7400da44529f2c3905e61'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.171659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '604eb9039ce7400da44529f2c3905e61'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.182044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '604eb9039ce7400da44529f2c3905e61'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.183064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '604eb9039ce7400da44529f2c3905e61'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.268369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1008/2010] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:44 2022] DELETE /identity/v3/users/604eb9039ce7400da44529f2c3905e61 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:45.272775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c97328-e3f6-414e-861b-f108bf0d1952 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:45.273493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c97328-e3f6-414e-861b-f108bf0d1952 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:45.273493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c97328-e3f6-414e-861b-f108bf0d1952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:45.316109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-02c97328-e3f6-414e-861b-f108bf0d1952 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:45.317616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-02c97328-e3f6-414e-861b-f108bf0d1952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:45.498100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1003/2011] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:45.601885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:45.604065 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:45.719605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74cf57-42bf-4f8f-b2ea-6c29788aafbb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:45.726744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061a710-3935-450a-94ee-f0335042fed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:45.728672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061a710-3935-450a-94ee-f0335042fed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:45.729182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061a710-3935-450a-94ee-f0335042fed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:45.729679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061a710-3935-450a-94ee-f0335042fed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:45.757354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061a710-3935-450a-94ee-f0335042fed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:45.764964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061a710-3935-450a-94ee-f0335042fed3 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:45.801923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1009/2012] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:46.538581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c97328-e3f6-414e-861b-f108bf0d1952 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:46.542277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-02c97328-e3f6-414e-861b-f108bf0d1952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:46.587100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c97328-e3f6-414e-861b-f108bf0d1952 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:46.596340 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab784207-b582-472a-965f-46686996acea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:46.598790 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab784207-b582-472a-965f-46686996acea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:46.599539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab784207-b582-472a-965f-46686996acea admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:46.600636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab784207-b582-472a-965f-46686996acea admin admin] PATH_INFO: `/v3/projects/832843c38216472dae48b1f446e8a2f9` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:46.605414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab784207-b582-472a-965f-46686996acea admin admin] RBAC: Authorizing `identity:delete_project(project_id=832843c38216472dae48b1f446e8a2f9)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:46.609952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab784207-b582-472a-965f-46686996acea admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:46.745181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ab784207-b582-472a-965f-46686996acea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '832843c38216472dae48b1f446e8a2f9', 'request_id': 'req-ab784207-b582-472a-965f-46686996acea'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:46.757723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ab784207-b582-472a-965f-46686996acea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '832843c38216472dae48b1f446e8a2f9', 'request_id': 'req-ab784207-b582-472a-965f-46686996acea'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:46.779975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ab784207-b582-472a-965f-46686996acea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '832843c38216472dae48b1f446e8a2f9', 'request_id': 'req-ab784207-b582-472a-965f-46686996acea'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:46.814284 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ab784207-b582-472a-965f-46686996acea admin admin] The token cache is being invalidate because project 832843c38216472dae48b1f446e8a2f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:46.814585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ab784207-b582-472a-965f-46686996acea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:46.817136 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ab784207-b582-472a-965f-46686996acea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:46.822530 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1004/2013] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:46 2022] DELETE /identity/v3/projects/832843c38216472dae48b1f446e8a2f9 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:46.826987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061a710-3935-450a-94ee-f0335042fed3 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:46.853606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061a710-3935-450a-94ee-f0335042fed3 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:46.915452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:47.008508 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ab31f-2f5a-4a36-ad2a-cd68e70916ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:47.011350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ab31f-2f5a-4a36-ad2a-cd68e70916ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:47.011887 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ab31f-2f5a-4a36-ad2a-cd68e70916ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:47.012398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ab31f-2f5a-4a36-ad2a-cd68e70916ea admin admin] PATH_INFO: `/v3/users/b2c2304cac3242149be6132a4b101ada` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:47.014663 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ab31f-2f5a-4a36-ad2a-cd68e70916ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c2304cac3242149be6132a4b101ada)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:47.018460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ab31f-2f5a-4a36-ad2a-cd68e70916ea admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:47.106578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cb5ab31f-2f5a-4a36-ad2a-cd68e70916ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c2304cac3242149be6132a4b101ada'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:47.125089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cb5ab31f-2f5a-4a36-ad2a-cd68e70916ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c2304cac3242149be6132a4b101ada'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:47.125919 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cb5ab31f-2f5a-4a36-ad2a-cd68e70916ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c2304cac3242149be6132a4b101ada'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:47.142761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cb5ab31f-2f5a-4a36-ad2a-cd68e70916ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c2304cac3242149be6132a4b101ada'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:47.143957 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cb5ab31f-2f5a-4a36-ad2a-cd68e70916ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c2304cac3242149be6132a4b101ada'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:47.172877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1010/2014] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:46 2022] DELETE /identity/v3/users/b2c2304cac3242149be6132a4b101ada => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:47.549654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab784207-b582-472a-965f-46686996acea admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:47.552767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ab784207-b582-472a-965f-46686996acea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:47.597037 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab784207-b582-472a-965f-46686996acea admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:47.704258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b0317-928c-4adb-a93b-f76590f7b845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:47.705436 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b0317-928c-4adb-a93b-f76590f7b845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:47.705882 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b0317-928c-4adb-a93b-f76590f7b845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:47.706458 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b0317-928c-4adb-a93b-f76590f7b845 admin admin] PATH_INFO: `/v3/projects/069d02cc804a42c987d006b779ddcb3b` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:47.709483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b0317-928c-4adb-a93b-f76590f7b845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=069d02cc804a42c987d006b779ddcb3b)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:47.712968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b0317-928c-4adb-a93b-f76590f7b845 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:47.772841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:47.817947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e37b0317-928c-4adb-a93b-f76590f7b845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '069d02cc804a42c987d006b779ddcb3b', 'request_id': 'req-e37b0317-928c-4adb-a93b-f76590f7b845'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:47.831442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e37b0317-928c-4adb-a93b-f76590f7b845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '069d02cc804a42c987d006b779ddcb3b', 'request_id': 'req-e37b0317-928c-4adb-a93b-f76590f7b845'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:47.844034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e37b0317-928c-4adb-a93b-f76590f7b845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '069d02cc804a42c987d006b779ddcb3b', 'request_id': 'req-e37b0317-928c-4adb-a93b-f76590f7b845'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:47.866217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e37b0317-928c-4adb-a93b-f76590f7b845 admin admin] The token cache is being invalidate because project 069d02cc804a42c987d006b779ddcb3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:47.866654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e37b0317-928c-4adb-a93b-f76590f7b845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:47.867606 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e37b0317-928c-4adb-a93b-f76590f7b845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:47.871197 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1005/2015] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:47 2022] DELETE /identity/v3/projects/069d02cc804a42c987d006b779ddcb3b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:47.878535 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1b9a2-4dda-4a65-9aad-6a950f335051 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:47.879336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1b9a2-4dda-4a65-9aad-6a950f335051 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:47.879558 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1b9a2-4dda-4a65-9aad-6a950f335051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:47.903607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-eed1b9a2-4dda-4a65-9aad-6a950f335051 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:47.905556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-eed1b9a2-4dda-4a65-9aad-6a950f335051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:47.977365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1011/2016] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:47.985899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcee88d-5be8-4989-8beb-285a4f70a300 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:47.986239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcee88d-5be8-4989-8beb-285a4f70a300 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:47.986563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcee88d-5be8-4989-8beb-285a4f70a300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:48.007602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-adcee88d-5be8-4989-8beb-285a4f70a300 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:48.009020 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-adcee88d-5be8-4989-8beb-285a4f70a300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:48.018454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1006/2017] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:48.023029 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1b9a2-4dda-4a65-9aad-6a950f335051 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:48.034198 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1b9a2-4dda-4a65-9aad-6a950f335051 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:48.039492 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a6aaf-361c-4431-8fb5-3057da99f8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:48.040789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a6aaf-361c-4431-8fb5-3057da99f8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:48.041110 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a6aaf-361c-4431-8fb5-3057da99f8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:48.041426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a6aaf-361c-4431-8fb5-3057da99f8d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:48.042217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a6aaf-361c-4431-8fb5-3057da99f8d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:48.042563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a6aaf-361c-4431-8fb5-3057da99f8d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:48.044307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a6aaf-361c-4431-8fb5-3057da99f8d4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:48.058416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1012/2018] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:08:48.063155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcee88d-5be8-4989-8beb-285a4f70a300 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:48.073560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcee88d-5be8-4989-8beb-285a4f70a300 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:48.079542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbc048-b904-443a-bcb8-f0f7005d42aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:48.080228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbc048-b904-443a-bcb8-f0f7005d42aa admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:48.080514 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbc048-b904-443a-bcb8-f0f7005d42aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:48.080796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbc048-b904-443a-bcb8-f0f7005d42aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:48.081501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbc048-b904-443a-bcb8-f0f7005d42aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:48.083462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbc048-b904-443a-bcb8-f0f7005d42aa admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:48.115442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1007/2019] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:48 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:48.116855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a6aaf-361c-4431-8fb5-3057da99f8d4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:48.129395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a6aaf-361c-4431-8fb5-3057da99f8d4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:48.157578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:48.214159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894bfbc-073d-4f8f-9e41-182769479220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:48.215085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894bfbc-073d-4f8f-9e41-182769479220 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:48.215420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894bfbc-073d-4f8f-9e41-182769479220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:48.215762 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894bfbc-073d-4f8f-9e41-182769479220 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:48.216506 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894bfbc-073d-4f8f-9e41-182769479220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:48.218634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894bfbc-073d-4f8f-9e41-182769479220 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:48.253724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1013/2020] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:48 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:48.258739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbc048-b904-443a-bcb8-f0f7005d42aa admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:48.281066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbc048-b904-443a-bcb8-f0f7005d42aa admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:48.292455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e36d20-70d1-4f97-b383-eae686e0e60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:48.293320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e36d20-70d1-4f97-b383-eae686e0e60d admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:48.293718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e36d20-70d1-4f97-b383-eae686e0e60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:48.294141 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e36d20-70d1-4f97-b383-eae686e0e60d admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:48.295142 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e36d20-70d1-4f97-b383-eae686e0e60d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:48.297252 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e36d20-70d1-4f97-b383-eae686e0e60d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:48.314112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1008/2021] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:48.317792 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894bfbc-073d-4f8f-9e41-182769479220 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:48.331326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894bfbc-073d-4f8f-9e41-182769479220 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:48.338682 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081c0af-1de6-4e80-8e6c-abfb07908609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:48.340545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081c0af-1de6-4e80-8e6c-abfb07908609 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:48.340545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081c0af-1de6-4e80-8e6c-abfb07908609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:48.341129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081c0af-1de6-4e80-8e6c-abfb07908609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:48.342377 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081c0af-1de6-4e80-8e6c-abfb07908609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:48.345275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081c0af-1de6-4e80-8e6c-abfb07908609 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:48.364868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1014/2022] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:48.370660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e36d20-70d1-4f97-b383-eae686e0e60d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:48.391000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e36d20-70d1-4f97-b383-eae686e0e60d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:48.399915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca021d-bfa7-4e7d-a501-796b5d8bda73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:48.401525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca021d-bfa7-4e7d-a501-796b5d8bda73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:48.402033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca021d-bfa7-4e7d-a501-796b5d8bda73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:48.402545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca021d-bfa7-4e7d-a501-796b5d8bda73 admin admin] PATH_INFO: `/v3/projects/6d40780d34214bc6974bcb658be6b3d2/users/99127dff61834187834bff45137b5747/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:48.432334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca021d-bfa7-4e7d-a501-796b5d8bda73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d40780d34214bc6974bcb658be6b3d2, user_id=99127dff61834187834bff45137b5747, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:48.437583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca021d-bfa7-4e7d-a501-796b5d8bda73 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:48.457891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1009/2023] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:48 2022] PUT /identity/v3/projects/6d40780d34214bc6974bcb658be6b3d2/users/99127dff61834187834bff45137b5747/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:08:48.472024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c81b19-f4bd-4c4a-880e-b4f15fb87861 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:48.472685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c81b19-f4bd-4c4a-880e-b4f15fb87861 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:48.473672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c81b19-f4bd-4c4a-880e-b4f15fb87861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:48.518829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b8c81b19-f4bd-4c4a-880e-b4f15fb87861 None None] MFA Rules not processed for user `99127dff61834187834bff45137b5747`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:48.520148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b8c81b19-f4bd-4c4a-880e-b4f15fb87861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:48.611547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1015/2024] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:48.623917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca021d-bfa7-4e7d-a501-796b5d8bda73 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:48.626030 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ffca021d-bfa7-4e7d-a501-796b5d8bda73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:48.690414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca021d-bfa7-4e7d-a501-796b5d8bda73 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:48.694496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaf066-e0cd-4b80-b840-110b04a6d50e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:48.695161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaf066-e0cd-4b80-b840-110b04a6d50e service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:48.695486 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaf066-e0cd-4b80-b840-110b04a6d50e service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:48.695773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaf066-e0cd-4b80-b840-110b04a6d50e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:48.705415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfaf066-e0cd-4b80-b840-110b04a6d50e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:48.706917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfaf066-e0cd-4b80-b840-110b04a6d50e service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:48.717008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1010/2025] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:58.607574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:58.712754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:58.771819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896924d-b008-46f8-a7ac-ee53070857e3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:58.772118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896924d-b008-46f8-a7ac-ee53070857e3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:58.772403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896924d-b008-46f8-a7ac-ee53070857e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:58.791737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d896924d-b008-46f8-a7ac-ee53070857e3 None None] MFA Rules not processed for user `99127dff61834187834bff45137b5747`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:58.793054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d896924d-b008-46f8-a7ac-ee53070857e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:58.800569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1016/2026] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:58.876420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaf066-e0cd-4b80-b840-110b04a6d50e service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:58.879131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9bfaf066-e0cd-4b80-b840-110b04a6d50e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:58.963047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaf066-e0cd-4b80-b840-110b04a6d50e service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:58.967740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:58.968761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:58.969047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:58.969690 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] PATH_INFO: `/v3/users/1262f95bcaa24ff69e65ecb88a21f711` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:58.970516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1262f95bcaa24ff69e65ecb88a21f711)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:58.973539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:59.028457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1262f95bcaa24ff69e65ecb88a21f711'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:59.029134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1262f95bcaa24ff69e65ecb88a21f711'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:59.037761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1262f95bcaa24ff69e65ecb88a21f711'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:59.049471 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1262f95bcaa24ff69e65ecb88a21f711'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:59.050314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1262f95bcaa24ff69e65ecb88a21f711'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:59.074987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1011/2027] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:58 2022] DELETE /identity/v3/users/1262f95bcaa24ff69e65ecb88a21f711 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:59.079366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08772c74-b6fd-4230-85a8-2178ae8db3c0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:59.080381 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08772c74-b6fd-4230-85a8-2178ae8db3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:59.080381 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08772c74-b6fd-4230-85a8-2178ae8db3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:59.098292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-08772c74-b6fd-4230-85a8-2178ae8db3c0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:59.099522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-08772c74-b6fd-4230-85a8-2178ae8db3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:59.179502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1017/2028] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:59.189227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:59.201709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:59.276738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82172ee3-2964-486c-941c-ce8e98b8953a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:59.292720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e463b7e-db03-4806-858d-d1ca743367bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:59.293545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e463b7e-db03-4806-858d-d1ca743367bf service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:59.293887 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e463b7e-db03-4806-858d-d1ca743367bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:59.294225 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e463b7e-db03-4806-858d-d1ca743367bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:59.306491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e463b7e-db03-4806-858d-d1ca743367bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:59.307942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e463b7e-db03-4806-858d-d1ca743367bf service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:59.322333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1012/2029] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:59.847025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08772c74-b6fd-4230-85a8-2178ae8db3c0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:59.852930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-08772c74-b6fd-4230-85a8-2178ae8db3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:59.872648 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08772c74-b6fd-4230-85a8-2178ae8db3c0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:59.894373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f705b-f6e5-44ec-8db5-3d54ff87520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:59.895891 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f705b-f6e5-44ec-8db5-3d54ff87520f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:59.896374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f705b-f6e5-44ec-8db5-3d54ff87520f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:59.897855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f705b-f6e5-44ec-8db5-3d54ff87520f admin admin] PATH_INFO: `/v3/projects/758d2f7671144774a12542f664f2f028` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:59.900138 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f705b-f6e5-44ec-8db5-3d54ff87520f admin admin] RBAC: Authorizing `identity:delete_project(project_id=758d2f7671144774a12542f664f2f028)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:59.903249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f705b-f6e5-44ec-8db5-3d54ff87520f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:00.001290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-056f705b-f6e5-44ec-8db5-3d54ff87520f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '758d2f7671144774a12542f664f2f028', 'request_id': 'req-056f705b-f6e5-44ec-8db5-3d54ff87520f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:00.015465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-056f705b-f6e5-44ec-8db5-3d54ff87520f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '758d2f7671144774a12542f664f2f028', 'request_id': 'req-056f705b-f6e5-44ec-8db5-3d54ff87520f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:00.065782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-056f705b-f6e5-44ec-8db5-3d54ff87520f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '758d2f7671144774a12542f664f2f028', 'request_id': 'req-056f705b-f6e5-44ec-8db5-3d54ff87520f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:00.093313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-056f705b-f6e5-44ec-8db5-3d54ff87520f admin admin] The token cache is being invalidate because project 758d2f7671144774a12542f664f2f028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:00.093722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-056f705b-f6e5-44ec-8db5-3d54ff87520f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:00.094731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-056f705b-f6e5-44ec-8db5-3d54ff87520f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:00.099248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1018/2030] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:59 2022] DELETE /identity/v3/projects/758d2f7671144774a12542f664f2f028 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:00.104372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8bef3-5882-4c37-bdc1-a0779e6bc15a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:00.105056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8bef3-5882-4c37-bdc1-a0779e6bc15a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:00.105056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8bef3-5882-4c37-bdc1-a0779e6bc15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:00.127859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-90f8bef3-5882-4c37-bdc1-a0779e6bc15a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:00.129414 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-90f8bef3-5882-4c37-bdc1-a0779e6bc15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:00.154256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:00.235629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1013/2031] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:00.244794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91186f-c9bc-4eaf-8b8e-587930b9a6a9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:00.245639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91186f-c9bc-4eaf-8b8e-587930b9a6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:00.246382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91186f-c9bc-4eaf-8b8e-587930b9a6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:00.283198 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4b91186f-c9bc-4eaf-8b8e-587930b9a6a9 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:00.284470 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4b91186f-c9bc-4eaf-8b8e-587930b9a6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:00.294532 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1019/2032] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:00.301207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8bef3-5882-4c37-bdc1-a0779e6bc15a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:00.311978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8bef3-5882-4c37-bdc1-a0779e6bc15a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:00.317146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696660c-e5cb-4a9c-b527-16d07f6cc58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:00.317839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696660c-e5cb-4a9c-b527-16d07f6cc58e admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:00.318128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696660c-e5cb-4a9c-b527-16d07f6cc58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:00.318408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696660c-e5cb-4a9c-b527-16d07f6cc58e admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:00.319143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696660c-e5cb-4a9c-b527-16d07f6cc58e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:09:00.319484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696660c-e5cb-4a9c-b527-16d07f6cc58e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:00.321727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696660c-e5cb-4a9c-b527-16d07f6cc58e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:00.340402 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1014/2033] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:09:00.344961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91186f-c9bc-4eaf-8b8e-587930b9a6a9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:00.356507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91186f-c9bc-4eaf-8b8e-587930b9a6a9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:00.362664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a825879-4608-499f-841b-5f45defbf91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:00.363763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a825879-4608-499f-841b-5f45defbf91f admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:00.364120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a825879-4608-499f-841b-5f45defbf91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:00.364625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a825879-4608-499f-841b-5f45defbf91f admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:00.365452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a825879-4608-499f-841b-5f45defbf91f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:00.367826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a825879-4608-499f-841b-5f45defbf91f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:00.403443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1020/2034] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:00 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:00.408498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696660c-e5cb-4a9c-b527-16d07f6cc58e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:00.425080 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696660c-e5cb-4a9c-b527-16d07f6cc58e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:00.531616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cee086-f571-49a6-bf2b-75aaf61f27d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:00.532413 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cee086-f571-49a6-bf2b-75aaf61f27d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:00.533033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cee086-f571-49a6-bf2b-75aaf61f27d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:00.533474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cee086-f571-49a6-bf2b-75aaf61f27d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:00.534283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cee086-f571-49a6-bf2b-75aaf61f27d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:00.539216 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cee086-f571-49a6-bf2b-75aaf61f27d6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:00.574263 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1015/2035] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:00 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:00.578109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a825879-4608-499f-841b-5f45defbf91f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:00.605563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a825879-4608-499f-841b-5f45defbf91f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:00.621054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0944129-5d84-451b-b2b9-ce62f2ff808f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:00.623548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0944129-5d84-451b-b2b9-ce62f2ff808f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:00.623548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0944129-5d84-451b-b2b9-ce62f2ff808f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:00.623838 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0944129-5d84-451b-b2b9-ce62f2ff808f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:00.624684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0944129-5d84-451b-b2b9-ce62f2ff808f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:00.626575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0944129-5d84-451b-b2b9-ce62f2ff808f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:00.638757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1021/2036] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:00.643004 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cee086-f571-49a6-bf2b-75aaf61f27d6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:00.658739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cee086-f571-49a6-bf2b-75aaf61f27d6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:00.667063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbc3a7-5d49-4c09-870f-a55df8158f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:00.667772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbc3a7-5d49-4c09-870f-a55df8158f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:00.668060 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbc3a7-5d49-4c09-870f-a55df8158f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:00.668875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbc3a7-5d49-4c09-870f-a55df8158f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:00.669102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbc3a7-5d49-4c09-870f-a55df8158f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:00.672191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbc3a7-5d49-4c09-870f-a55df8158f8f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:00.688706 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1016/2037] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:00.690122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0944129-5d84-451b-b2b9-ce62f2ff808f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:00.707906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0944129-5d84-451b-b2b9-ce62f2ff808f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:00.726802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56307e-6ebc-4e25-971f-1f1e3e12ac9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:00.727891 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56307e-6ebc-4e25-971f-1f1e3e12ac9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:00.728253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56307e-6ebc-4e25-971f-1f1e3e12ac9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:00.729198 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56307e-6ebc-4e25-971f-1f1e3e12ac9c admin admin] PATH_INFO: `/v3/projects/6f547d682d364b2789dd11bb434a89ef/users/d2624265a4b945099d024e826cf00123/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:00.747941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56307e-6ebc-4e25-971f-1f1e3e12ac9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f547d682d364b2789dd11bb434a89ef, user_id=d2624265a4b945099d024e826cf00123, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:00.750710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56307e-6ebc-4e25-971f-1f1e3e12ac9c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:00.767506 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1022/2038] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:00 2022] PUT /identity/v3/projects/6f547d682d364b2789dd11bb434a89ef/users/d2624265a4b945099d024e826cf00123/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:00.778714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07223775-420e-49ee-86f4-ccf0cc8150f6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:00.779011 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07223775-420e-49ee-86f4-ccf0cc8150f6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:00.779291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07223775-420e-49ee-86f4-ccf0cc8150f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:00.821072 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-07223775-420e-49ee-86f4-ccf0cc8150f6 None None] MFA Rules not processed for user `d2624265a4b945099d024e826cf00123`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:00.822326 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-07223775-420e-49ee-86f4-ccf0cc8150f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:00.907822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1017/2039] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:00.912685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56307e-6ebc-4e25-971f-1f1e3e12ac9c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:00.928753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:01.002904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56307e-6ebc-4e25-971f-1f1e3e12ac9c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:01.008603 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddddd36-e469-4c47-8850-e5a1ea4ad8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:01.009193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddddd36-e469-4c47-8850-e5a1ea4ad8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:01.009483 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddddd36-e469-4c47-8850-e5a1ea4ad8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:01.009769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddddd36-e469-4c47-8850-e5a1ea4ad8c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:01.010472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddddd36-e469-4c47-8850-e5a1ea4ad8c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:01.012601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddddd36-e469-4c47-8850-e5a1ea4ad8c3 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:01.061299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1023/2040] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:00 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:01.065349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07223775-420e-49ee-86f4-ccf0cc8150f6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:01.078232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07223775-420e-49ee-86f4-ccf0cc8150f6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:01.111793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:01.173726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bbaca-8f18-4173-be54-1218c5877405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:01.174682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bbaca-8f18-4173-be54-1218c5877405 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:01.175453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bbaca-8f18-4173-be54-1218c5877405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:01.175453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bbaca-8f18-4173-be54-1218c5877405 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:01.176168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70bbaca-8f18-4173-be54-1218c5877405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:01.179798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70bbaca-8f18-4173-be54-1218c5877405 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:01.226912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1018/2041] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:01 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:01.232154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddddd36-e469-4c47-8850-e5a1ea4ad8c3 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:01.250445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddddd36-e469-4c47-8850-e5a1ea4ad8c3 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:01.260190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5081220-031c-4525-afa0-169fdffae485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:01.261214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5081220-031c-4525-afa0-169fdffae485 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:01.263164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5081220-031c-4525-afa0-169fdffae485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:01.264671 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5081220-031c-4525-afa0-169fdffae485 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:01.265965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5081220-031c-4525-afa0-169fdffae485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:01.269601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5081220-031c-4525-afa0-169fdffae485 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:01.293573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1024/2042] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:01.297816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bbaca-8f18-4173-be54-1218c5877405 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:01.317310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bbaca-8f18-4173-be54-1218c5877405 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:01.326799 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556b2f3-1f18-4efb-84e6-7e03474af4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:01.327525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556b2f3-1f18-4efb-84e6-7e03474af4df admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:01.327816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556b2f3-1f18-4efb-84e6-7e03474af4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:01.328099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556b2f3-1f18-4efb-84e6-7e03474af4df admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:01.328825 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556b2f3-1f18-4efb-84e6-7e03474af4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:01.330606 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556b2f3-1f18-4efb-84e6-7e03474af4df admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:01.343985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1019/2043] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:09:01.348221 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5081220-031c-4525-afa0-169fdffae485 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:01.360973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5081220-031c-4525-afa0-169fdffae485 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:01.368367 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b53c64-70b5-4ef5-b5ea-e951fe084bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:01.369767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b53c64-70b5-4ef5-b5ea-e951fe084bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:01.369941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b53c64-70b5-4ef5-b5ea-e951fe084bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:01.370450 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b53c64-70b5-4ef5-b5ea-e951fe084bd7 admin admin] PATH_INFO: `/v3/projects/bb6f3f966dd342879dba1629382940bb/users/7a57ce9dbf324046b486758fddfaa309/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:01.397566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b53c64-70b5-4ef5-b5ea-e951fe084bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6f3f966dd342879dba1629382940bb, user_id=7a57ce9dbf324046b486758fddfaa309, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:01.400662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b53c64-70b5-4ef5-b5ea-e951fe084bd7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:01.431859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1025/2044] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:01 2022] PUT /identity/v3/projects/bb6f3f966dd342879dba1629382940bb/users/7a57ce9dbf324046b486758fddfaa309/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:01.442087 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb60a8-ba39-4ad0-ae1c-2985170e471c None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:01.442412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb60a8-ba39-4ad0-ae1c-2985170e471c None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:01.442663 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb60a8-ba39-4ad0-ae1c-2985170e471c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:01.504418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b4bb60a8-ba39-4ad0-ae1c-2985170e471c None None] MFA Rules not processed for user `7a57ce9dbf324046b486758fddfaa309`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:01.506174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b4bb60a8-ba39-4ad0-ae1c-2985170e471c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:01.659224 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b53c64-70b5-4ef5-b5ea-e951fe084bd7 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:01.661319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-59b53c64-70b5-4ef5-b5ea-e951fe084bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:01.666281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1020/2045] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:01.667467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb60a8-ba39-4ad0-ae1c-2985170e471c None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:01.820534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b53c64-70b5-4ef5-b5ea-e951fe084bd7 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:01.835062 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:01.836389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:01.837539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:01.838084 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] PATH_INFO: `/v3/users/94144004dab94e7aa60f33c308903b00` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:01.844107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94144004dab94e7aa60f33c308903b00)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:01.847119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:01.867342 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb60a8-ba39-4ad0-ae1c-2985170e471c None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:01.892108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db40dc9-226d-47ac-a26e-6494d47e2db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:01.892580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db40dc9-226d-47ac-a26e-6494d47e2db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:01.892870 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db40dc9-226d-47ac-a26e-6494d47e2db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:01.893153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db40dc9-226d-47ac-a26e-6494d47e2db1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:01.893865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db40dc9-226d-47ac-a26e-6494d47e2db1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:01.895897 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db40dc9-226d-47ac-a26e-6494d47e2db1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:01.958079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1021/2046] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:01 2022] POST /identity/v3/projects => generated 385 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:01.958079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db40dc9-226d-47ac-a26e-6494d47e2db1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:01.964589 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6db40dc9-226d-47ac-a26e-6494d47e2db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:01.972120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94144004dab94e7aa60f33c308903b00'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:01.983483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:01.985051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94144004dab94e7aa60f33c308903b00'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:01.985996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94144004dab94e7aa60f33c308903b00'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:01.996919 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94144004dab94e7aa60f33c308903b00'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:01.998108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94144004dab94e7aa60f33c308903b00'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:02.002703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:02.039434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1026/2047] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:01 2022] DELETE /identity/v3/users/94144004dab94e7aa60f33c308903b00 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:02.039434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:02.041049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:02.070417 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db40dc9-226d-47ac-a26e-6494d47e2db1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:02.135491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67713e-e182-4142-9cbd-7328523b5a71 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:02.143488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:02.144890 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e501d6-97f1-49bb-8e3a-36387570853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:02.145744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e501d6-97f1-49bb-8e3a-36387570853d admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.146128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e501d6-97f1-49bb-8e3a-36387570853d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.147056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e501d6-97f1-49bb-8e3a-36387570853d admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.147408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e501d6-97f1-49bb-8e3a-36387570853d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:02.147675 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.148725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.149788 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] PATH_INFO: `/v3/users/3f7fd9aa8fb147098c6a32c11017f79d` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.150361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e501d6-97f1-49bb-8e3a-36387570853d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:02.151588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f7fd9aa8fb147098c6a32c11017f79d)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:02.154382 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:02.179024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1027/2048] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:02 2022] POST /identity/v3/users => generated 299 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:02.184594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e84dc1-0681-44a7-97d6-ff75ae289191 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.184881 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e84dc1-0681-44a7-97d6-ff75ae289191 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.185179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e84dc1-0681-44a7-97d6-ff75ae289191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.224783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d2e84dc1-0681-44a7-97d6-ff75ae289191 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:02.226253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d2e84dc1-0681-44a7-97d6-ff75ae289191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:02.241274 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1028/2049] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:02.241856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e84dc1-0681-44a7-97d6-ff75ae289191 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:02.269716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f7fd9aa8fb147098c6a32c11017f79d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:02.270932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f7fd9aa8fb147098c6a32c11017f79d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:02.274616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e84dc1-0681-44a7-97d6-ff75ae289191 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:02.286600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f7fd9aa8fb147098c6a32c11017f79d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:02.288758 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd0003-47c7-40d0-b007-f9dab42d80b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:02.290356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd0003-47c7-40d0-b007-f9dab42d80b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.291475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd0003-47c7-40d0-b007-f9dab42d80b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.291960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd0003-47c7-40d0-b007-f9dab42d80b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.293583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd0003-47c7-40d0-b007-f9dab42d80b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:02.296960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd0003-47c7-40d0-b007-f9dab42d80b9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:02.302821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f7fd9aa8fb147098c6a32c11017f79d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:02.305016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f7fd9aa8fb147098c6a32c11017f79d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:02.314633 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1029/2050] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:02.315642 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd0003-47c7-40d0-b007-f9dab42d80b9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:02.317910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-86cd0003-47c7-40d0-b007-f9dab42d80b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:02.383917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1022/2051] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:01 2022] DELETE /identity/v3/users/3f7fd9aa8fb147098c6a32c11017f79d => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:02.384828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:02.390186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:02.401761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd0003-47c7-40d0-b007-f9dab42d80b9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:02.457334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:02.487126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9d03f-0858-44fc-8dbc-19133f37c80b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:02.496773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d7a8f-58c8-400c-b281-ecf16fb723a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:02.502396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d7a8f-58c8-400c-b281-ecf16fb723a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.502396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d7a8f-58c8-400c-b281-ecf16fb723a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.502396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d7a8f-58c8-400c-b281-ecf16fb723a8 admin admin] PATH_INFO: `/v3/projects/c90ba2f05e444c9c93925b20ee9c9a2c/users/a8ece0820a684704a3c01d0f5af63069/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.511911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9987ed-88e2-48ef-935f-d916b9b730fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:02.512591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9987ed-88e2-48ef-935f-d916b9b730fe service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.512876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9987ed-88e2-48ef-935f-d916b9b730fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.513156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9987ed-88e2-48ef-935f-d916b9b730fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.516353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1a9987ed-88e2-48ef-935f-d916b9b730fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:02.530842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d7a8f-58c8-400c-b281-ecf16fb723a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90ba2f05e444c9c93925b20ee9c9a2c, user_id=a8ece0820a684704a3c01d0f5af63069, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:02.534364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d7a8f-58c8-400c-b281-ecf16fb723a8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:02.537293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9987ed-88e2-48ef-935f-d916b9b730fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:02.538834 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9987ed-88e2-48ef-935f-d916b9b730fe service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:02.558241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1023/2052] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:02 2022] PUT /identity/v3/projects/c90ba2f05e444c9c93925b20ee9c9a2c/users/a8ece0820a684704a3c01d0f5af63069/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:02.559776 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b795b6-a16f-4639-9c06-974061b76c0b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.560074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b795b6-a16f-4639-9c06-974061b76c0b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.560363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b795b6-a16f-4639-9c06-974061b76c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.561273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1030/2053] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:02.562197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9987ed-88e2-48ef-935f-d916b9b730fe service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:02.575201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9987ed-88e2-48ef-935f-d916b9b730fe service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:02.586811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:02.600540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c7b795b6-a16f-4639-9c06-974061b76c0b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:02.605019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c7b795b6-a16f-4639-9c06-974061b76c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:02.666447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c13890-d6b8-48a2-9131-0cc916028abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:02.667155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c13890-d6b8-48a2-9131-0cc916028abd admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.667505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c13890-d6b8-48a2-9131-0cc916028abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.668579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c13890-d6b8-48a2-9131-0cc916028abd admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.669351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c13890-d6b8-48a2-9131-0cc916028abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:02.671701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c13890-d6b8-48a2-9131-0cc916028abd admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:02.692982 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1031/2054] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:02.694138 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c13890-d6b8-48a2-9131-0cc916028abd admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:02.709494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c13890-d6b8-48a2-9131-0cc916028abd admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:02.716212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580c8ac-6a6c-4abd-b09d-7acf113855bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:02.717208 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580c8ac-6a6c-4abd-b09d-7acf113855bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.717496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580c8ac-6a6c-4abd-b09d-7acf113855bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.717829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580c8ac-6a6c-4abd-b09d-7acf113855bb admin admin] PATH_INFO: `/v3/projects/c90ba2f05e444c9c93925b20ee9c9a2c/users/a8ece0820a684704a3c01d0f5af63069/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.720469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580c8ac-6a6c-4abd-b09d-7acf113855bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90ba2f05e444c9c93925b20ee9c9a2c, user_id=a8ece0820a684704a3c01d0f5af63069, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:02.724064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580c8ac-6a6c-4abd-b09d-7acf113855bb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:02.743732 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1032/2055] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:02 2022] PUT /identity/v3/projects/c90ba2f05e444c9c93925b20ee9c9a2c/users/a8ece0820a684704a3c01d0f5af63069/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:02.753542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c4501-ae38-43b0-8e78-2e6998492f5b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:02.754360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c4501-ae38-43b0-8e78-2e6998492f5b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:02.754953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c4501-ae38-43b0-8e78-2e6998492f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:02.767350 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1024/2056] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:02.802272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b32c4501-ae38-43b0-8e78-2e6998492f5b None None] MFA Rules not processed for user `a8ece0820a684704a3c01d0f5af63069`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:02.803893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b32c4501-ae38-43b0-8e78-2e6998492f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:02.806641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b795b6-a16f-4639-9c06-974061b76c0b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:02.949517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1033/2057] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:02.959620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c4501-ae38-43b0-8e78-2e6998492f5b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:02.960019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b795b6-a16f-4639-9c06-974061b76c0b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:02.961292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b32c4501-ae38-43b0-8e78-2e6998492f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:03.163932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6bdb2-dbb7-445a-a92f-58475f58abf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:03.167249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6bdb2-dbb7-445a-a92f-58475f58abf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:03.167844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6bdb2-dbb7-445a-a92f-58475f58abf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:03.168237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6bdb2-dbb7-445a-a92f-58475f58abf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:03.181642 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c4501-ae38-43b0-8e78-2e6998492f5b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:03.199666 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ad386-349d-4336-88d5-6ab868b105cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:03.201322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ad386-349d-4336-88d5-6ab868b105cb service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:03.201780 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ad386-349d-4336-88d5-6ab868b105cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:03.202262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ad386-349d-4336-88d5-6ab868b105cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:03.205397 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-634ad386-349d-4336-88d5-6ab868b105cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:03.277885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:03.310815 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:03.332200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6bdb2-dbb7-445a-a92f-58475f58abf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:03.343995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6bdb2-dbb7-445a-a92f-58475f58abf5 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:03.354253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ad386-349d-4336-88d5-6ab868b105cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:03.356408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ad386-349d-4336-88d5-6ab868b105cb service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:03.359311 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1025/2058] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:03.360455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6bdb2-dbb7-445a-a92f-58475f58abf5 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:03.361944 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-eef6bdb2-dbb7-445a-a92f-58475f58abf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:03.370170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1034/2059] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:03.377003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6bdb2-dbb7-445a-a92f-58475f58abf5 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:03.381873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:03.382797 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:03.383082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:03.383429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d admin admin] PATH_INFO: `/v3/projects/8bf3d02b9d8d431b9b1a76dba93f77e2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:03.385101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf3d02b9d8d431b9b1a76dba93f77e2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:03.387818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:03.480116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf3d02b9d8d431b9b1a76dba93f77e2', 'request_id': 'req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:03.487738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf3d02b9d8d431b9b1a76dba93f77e2', 'request_id': 'req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:03.494476 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf3d02b9d8d431b9b1a76dba93f77e2', 'request_id': 'req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:03.524991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d admin admin] The token cache is being invalidate because project 8bf3d02b9d8d431b9b1a76dba93f77e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:03.525297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:03.528018 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a8cbfe09-8a6d-4ea2-b077-bc880fbae14d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:03.530448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1026/2060] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:03 2022] DELETE /identity/v3/projects/8bf3d02b9d8d431b9b1a76dba93f77e2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:03.535691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4644c-a7fe-48af-aeef-20ab8f0a5a72 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:03.536874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4644c-a7fe-48af-aeef-20ab8f0a5a72 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:03.536874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4644c-a7fe-48af-aeef-20ab8f0a5a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:03.569321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-81a4644c-a7fe-48af-aeef-20ab8f0a5a72 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:03.573793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-81a4644c-a7fe-48af-aeef-20ab8f0a5a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:03.709107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1035/2061] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:03.723036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452b70b-e8d0-4194-ae19-acc4117c78ee None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:03.723331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452b70b-e8d0-4194-ae19-acc4117c78ee None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:03.723653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452b70b-e8d0-4194-ae19-acc4117c78ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:03.761259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e452b70b-e8d0-4194-ae19-acc4117c78ee None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:03.762686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e452b70b-e8d0-4194-ae19-acc4117c78ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:03.785100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1027/2062] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:03.789511 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4644c-a7fe-48af-aeef-20ab8f0a5a72 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:03.807929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4644c-a7fe-48af-aeef-20ab8f0a5a72 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:03.826940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe5f5f-2c97-45d8-8f5a-1ce3e89c1f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:03.827832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe5f5f-2c97-45d8-8f5a-1ce3e89c1f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:03.828125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe5f5f-2c97-45d8-8f5a-1ce3e89c1f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:03.828420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe5f5f-2c97-45d8-8f5a-1ce3e89c1f30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:03.832985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe5f5f-2c97-45d8-8f5a-1ce3e89c1f30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:09:03.833650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe5f5f-2c97-45d8-8f5a-1ce3e89c1f30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:03.837774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe5f5f-2c97-45d8-8f5a-1ce3e89c1f30 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:03.871641 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1036/2063] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:09:03.876314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452b70b-e8d0-4194-ae19-acc4117c78ee None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:03.893431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452b70b-e8d0-4194-ae19-acc4117c78ee None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:03.915707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47cce2f-0add-4cae-970c-f78724625325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:03.918251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47cce2f-0add-4cae-970c-f78724625325 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:03.918795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47cce2f-0add-4cae-970c-f78724625325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:03.919686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47cce2f-0add-4cae-970c-f78724625325 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:03.941084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47cce2f-0add-4cae-970c-f78724625325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:03.950000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47cce2f-0add-4cae-970c-f78724625325 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:04.060586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1028/2064] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:03 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:04.061886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe5f5f-2c97-45d8-8f5a-1ce3e89c1f30 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:04.082653 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe5f5f-2c97-45d8-8f5a-1ce3e89c1f30 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:04.306710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbd23d-f548-4023-938c-59b263881086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:04.310124 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbd23d-f548-4023-938c-59b263881086 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:04.310604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbd23d-f548-4023-938c-59b263881086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:04.311092 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbd23d-f548-4023-938c-59b263881086 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:04.318847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbd23d-f548-4023-938c-59b263881086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:04.326191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbd23d-f548-4023-938c-59b263881086 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:04.371667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1037/2065] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:04 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:04.383313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47cce2f-0add-4cae-970c-f78724625325 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:04.426582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47cce2f-0add-4cae-970c-f78724625325 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:04.445346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4258afb-3c2f-4174-8ec8-49ced71652b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:04.446782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4258afb-3c2f-4174-8ec8-49ced71652b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:04.447069 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4258afb-3c2f-4174-8ec8-49ced71652b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:04.447421 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4258afb-3c2f-4174-8ec8-49ced71652b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:04.448239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4258afb-3c2f-4174-8ec8-49ced71652b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:04.449925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4258afb-3c2f-4174-8ec8-49ced71652b7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:04.465059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1029/2066] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:04.470024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbd23d-f548-4023-938c-59b263881086 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:04.484236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbd23d-f548-4023-938c-59b263881086 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:04.502068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d06d81-e3e3-47d5-9ad7-f8371b0d0873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:04.502772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d06d81-e3e3-47d5-9ad7-f8371b0d0873 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:04.503060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d06d81-e3e3-47d5-9ad7-f8371b0d0873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:04.503338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d06d81-e3e3-47d5-9ad7-f8371b0d0873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:04.504404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d06d81-e3e3-47d5-9ad7-f8371b0d0873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:04.506074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d06d81-e3e3-47d5-9ad7-f8371b0d0873 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:04.525520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4258afb-3c2f-4174-8ec8-49ced71652b7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:04.526336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1038/2067] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:04.540452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4258afb-3c2f-4174-8ec8-49ced71652b7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:04.553890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689666f0-0d5d-47e4-9975-06b0def6dff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:04.555322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689666f0-0d5d-47e4-9975-06b0def6dff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:04.556087 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689666f0-0d5d-47e4-9975-06b0def6dff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:04.556516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689666f0-0d5d-47e4-9975-06b0def6dff3 admin admin] PATH_INFO: `/v3/projects/8db2c4fd8b9840a2bf21107dbb291266/users/b283bdb8b4e94cdc9bd3e0212f09bce6/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:04.583032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689666f0-0d5d-47e4-9975-06b0def6dff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db2c4fd8b9840a2bf21107dbb291266, user_id=b283bdb8b4e94cdc9bd3e0212f09bce6, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:04.585925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689666f0-0d5d-47e4-9975-06b0def6dff3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:04.635435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1030/2068] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:04 2022] PUT /identity/v3/projects/8db2c4fd8b9840a2bf21107dbb291266/users/b283bdb8b4e94cdc9bd3e0212f09bce6/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:04.644121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15b11d-c080-4a8b-8331-f8a1dff4cacc None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:04.644415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15b11d-c080-4a8b-8331-f8a1dff4cacc None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:04.644695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15b11d-c080-4a8b-8331-f8a1dff4cacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:04.678666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689666f0-0d5d-47e4-9975-06b0def6dff3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:04.682510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-af15b11d-c080-4a8b-8331-f8a1dff4cacc None None] MFA Rules not processed for user `b283bdb8b4e94cdc9bd3e0212f09bce6`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:04.683173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-af15b11d-c080-4a8b-8331-f8a1dff4cacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:04.685622 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-689666f0-0d5d-47e4-9975-06b0def6dff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:04.739668 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:04.772807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:04.804432 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689666f0-0d5d-47e4-9975-06b0def6dff3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:04.814995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1039/2069] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:04.833117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15b11d-c080-4a8b-8331-f8a1dff4cacc None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:04.836075 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:04.836419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-af15b11d-c080-4a8b-8331-f8a1dff4cacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:04.837508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:04.838092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:04.838610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] PATH_INFO: `/v3/projects/02024f97891d489c8db7dac94182fe5d` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:04.841869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02024f97891d489c8db7dac94182fe5d)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:04.845847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:04.918621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15b11d-c080-4a8b-8331-f8a1dff4cacc None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:04.934149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02024f97891d489c8db7dac94182fe5d', 'request_id': 'req-eb66a308-1676-4c95-b7b4-e8a4224a5691'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:04.951720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02024f97891d489c8db7dac94182fe5d', 'request_id': 'req-eb66a308-1676-4c95-b7b4-e8a4224a5691'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:04.963368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02024f97891d489c8db7dac94182fe5d', 'request_id': 'req-eb66a308-1676-4c95-b7b4-e8a4224a5691'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:04.993774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e17e7f-b940-41d2-b4d7-2bdfc3bff409 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:04.994458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e17e7f-b940-41d2-b4d7-2bdfc3bff409 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:04.994779 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e17e7f-b940-41d2-b4d7-2bdfc3bff409 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:04.995061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e17e7f-b940-41d2-b4d7-2bdfc3bff409 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:04.996817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] The token cache is being invalidate because project 02024f97891d489c8db7dac94182fe5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:04.997197 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:05.000514 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:05.005961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1031/2070] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:04 2022] DELETE /identity/v3/projects/02024f97891d489c8db7dac94182fe5d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:05.015512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:05.020774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:05.120001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e17e7f-b940-41d2-b4d7-2bdfc3bff409 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:05.122811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e17e7f-b940-41d2-b4d7-2bdfc3bff409 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:05.155493 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1040/2071] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:05.180697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66a308-1676-4c95-b7b4-e8a4224a5691 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:05.194550 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:05.195614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:05.196699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:05.197083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] PATH_INFO: `/v3/users/f7f073d54b7b47949cd5125a069971de` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:05.199621 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f073d54b7b47949cd5125a069971de)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:05.205236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:05.305084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f073d54b7b47949cd5125a069971de'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:05.306526 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f073d54b7b47949cd5125a069971de'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:05.329539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f073d54b7b47949cd5125a069971de'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:05.329539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f073d54b7b47949cd5125a069971de'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:05.329539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f073d54b7b47949cd5125a069971de'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:05.353429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:05.381208 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1032/2072] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:05 2022] DELETE /identity/v3/users/f7f073d54b7b47949cd5125a069971de => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:05.430845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e17e7f-b940-41d2-b4d7-2bdfc3bff409 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:05.433697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e4e17e7f-b940-41d2-b4d7-2bdfc3bff409 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:05.465269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e17e7f-b940-41d2-b4d7-2bdfc3bff409 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:05.599521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:05.630476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfeb61-0f0d-4262-8449-d791d80a1f2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:05.631733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfeb61-0f0d-4262-8449-d791d80a1f2b service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:05.634351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfeb61-0f0d-4262-8449-d791d80a1f2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:05.635632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfeb61-0f0d-4262-8449-d791d80a1f2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:05.640816 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7bcfeb61-0f0d-4262-8449-d791d80a1f2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:05.828362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfeb61-0f0d-4262-8449-d791d80a1f2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:05.830088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfeb61-0f0d-4262-8449-d791d80a1f2b service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:05.855335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1041/2073] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:06.103695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:06.103695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:06.200126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f8877-a86a-4e74-9aa5-f13d162aa980 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:06.210909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24372b75-4acc-4a27-9158-ac801f3c549b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:06.212684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24372b75-4acc-4a27-9158-ac801f3c549b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:06.213203 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24372b75-4acc-4a27-9158-ac801f3c549b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:06.213562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24372b75-4acc-4a27-9158-ac801f3c549b admin admin] PATH_INFO: `/v3/projects/c5e987f9bcbe4d5db246a41b8d98f09f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:06.215171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24372b75-4acc-4a27-9158-ac801f3c549b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e987f9bcbe4d5db246a41b8d98f09f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:06.217323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24372b75-4acc-4a27-9158-ac801f3c549b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:06.334639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-24372b75-4acc-4a27-9158-ac801f3c549b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e987f9bcbe4d5db246a41b8d98f09f', 'request_id': 'req-24372b75-4acc-4a27-9158-ac801f3c549b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:06.354163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-24372b75-4acc-4a27-9158-ac801f3c549b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e987f9bcbe4d5db246a41b8d98f09f', 'request_id': 'req-24372b75-4acc-4a27-9158-ac801f3c549b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:06.372644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-24372b75-4acc-4a27-9158-ac801f3c549b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e987f9bcbe4d5db246a41b8d98f09f', 'request_id': 'req-24372b75-4acc-4a27-9158-ac801f3c549b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:06.420035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-24372b75-4acc-4a27-9158-ac801f3c549b admin admin] The token cache is being invalidate because project c5e987f9bcbe4d5db246a41b8d98f09f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:06.420637 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-24372b75-4acc-4a27-9158-ac801f3c549b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:06.422146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-24372b75-4acc-4a27-9158-ac801f3c549b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:06.427420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1033/2074] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:06 2022] DELETE /identity/v3/projects/c5e987f9bcbe4d5db246a41b8d98f09f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:06.431948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bb7a9-8d03-492b-b87b-ae3c319f1711 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:06.432510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bb7a9-8d03-492b-b87b-ae3c319f1711 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:06.432846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bb7a9-8d03-492b-b87b-ae3c319f1711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:06.459165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e37bb7a9-8d03-492b-b87b-ae3c319f1711 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:06.461322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e37bb7a9-8d03-492b-b87b-ae3c319f1711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:06.563517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1042/2075] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:06.575731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185b2ce-6245-407b-ac3c-d39f95d1df19 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:06.576338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185b2ce-6245-407b-ac3c-d39f95d1df19 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:06.576648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185b2ce-6245-407b-ac3c-d39f95d1df19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:06.604230 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1185b2ce-6245-407b-ac3c-d39f95d1df19 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:06.605876 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1185b2ce-6245-407b-ac3c-d39f95d1df19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:06.615890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1034/2076] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:06.622914 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bb7a9-8d03-492b-b87b-ae3c319f1711 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:06.644505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bb7a9-8d03-492b-b87b-ae3c319f1711 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:06.656170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137279a7-388d-4791-835d-83a796a9cc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:06.657531 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137279a7-388d-4791-835d-83a796a9cc9a admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:06.658116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137279a7-388d-4791-835d-83a796a9cc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:06.658544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137279a7-388d-4791-835d-83a796a9cc9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:06.660018 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137279a7-388d-4791-835d-83a796a9cc9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:09:06.660429 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137279a7-388d-4791-835d-83a796a9cc9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:06.662965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137279a7-388d-4791-835d-83a796a9cc9a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:06.686205 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1043/2077] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:09:06.690365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185b2ce-6245-407b-ac3c-d39f95d1df19 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:06.701611 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185b2ce-6245-407b-ac3c-d39f95d1df19 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:06.709738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19059ba0-a831-43a7-a74f-a9153a741b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:06.710545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19059ba0-a831-43a7-a74f-a9153a741b62 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:06.710761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19059ba0-a831-43a7-a74f-a9153a741b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:06.711092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19059ba0-a831-43a7-a74f-a9153a741b62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:06.712865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19059ba0-a831-43a7-a74f-a9153a741b62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:06.716005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19059ba0-a831-43a7-a74f-a9153a741b62 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:06.754639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1035/2078] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:06 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:06.756390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137279a7-388d-4791-835d-83a796a9cc9a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:06.772258 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137279a7-388d-4791-835d-83a796a9cc9a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:06.860909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acd39f-ad63-4f9f-a9d8-e89050eb5a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:06.861620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acd39f-ad63-4f9f-a9d8-e89050eb5a52 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:06.861897 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acd39f-ad63-4f9f-a9d8-e89050eb5a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:06.862345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acd39f-ad63-4f9f-a9d8-e89050eb5a52 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:06.863053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acd39f-ad63-4f9f-a9d8-e89050eb5a52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:06.865075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acd39f-ad63-4f9f-a9d8-e89050eb5a52 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:06.888738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1044/2079] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:06 2022] POST /identity/v3/users => generated 283 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:06.892969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19059ba0-a831-43a7-a74f-a9153a741b62 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:06.904831 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19059ba0-a831-43a7-a74f-a9153a741b62 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:06.909773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a13f6-941f-4a65-92c8-2dea2748132e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:06.910962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a13f6-941f-4a65-92c8-2dea2748132e admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:06.910962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a13f6-941f-4a65-92c8-2dea2748132e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:06.911139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a13f6-941f-4a65-92c8-2dea2748132e admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:06.911963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a13f6-941f-4a65-92c8-2dea2748132e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:06.913591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a13f6-941f-4a65-92c8-2dea2748132e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:06.944921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1036/2080] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:09:06.945594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acd39f-ad63-4f9f-a9d8-e89050eb5a52 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:06.961572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acd39f-ad63-4f9f-a9d8-e89050eb5a52 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:06.970449 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22193a18-d063-4ece-9742-7b553b5391e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:06.971617 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22193a18-d063-4ece-9742-7b553b5391e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:06.972027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22193a18-d063-4ece-9742-7b553b5391e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:06.972413 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22193a18-d063-4ece-9742-7b553b5391e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:06.974562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22193a18-d063-4ece-9742-7b553b5391e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:06.977520 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22193a18-d063-4ece-9742-7b553b5391e2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:06.997443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1045/2081] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:06.998555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a13f6-941f-4a65-92c8-2dea2748132e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:07.013983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a13f6-941f-4a65-92c8-2dea2748132e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:07.026060 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb4d6c-42cc-4349-9c8b-2623c8cb5743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:07.027343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb4d6c-42cc-4349-9c8b-2623c8cb5743 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:07.029637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22193a18-d063-4ece-9742-7b553b5391e2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:07.031679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-22193a18-d063-4ece-9742-7b553b5391e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:07.031807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb4d6c-42cc-4349-9c8b-2623c8cb5743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:07.032149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb4d6c-42cc-4349-9c8b-2623c8cb5743 admin admin] PATH_INFO: `/v3/projects/9518beef1490422b91771ceed94a995b/users/77893616533b4ad48dd6ba9dc6f5aee2/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:07.041269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:07.063945 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb4d6c-42cc-4349-9c8b-2623c8cb5743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9518beef1490422b91771ceed94a995b, user_id=77893616533b4ad48dd6ba9dc6f5aee2, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:07.066906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb4d6c-42cc-4349-9c8b-2623c8cb5743 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:07.085766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1037/2082] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:06 2022] PUT /identity/v3/projects/9518beef1490422b91771ceed94a995b/users/77893616533b4ad48dd6ba9dc6f5aee2/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:07.105385 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e5e36-97a9-4e41-82cb-ba1f3fe44e33 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:07.106397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e5e36-97a9-4e41-82cb-ba1f3fe44e33 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:07.106871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e5e36-97a9-4e41-82cb-ba1f3fe44e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:07.139234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22193a18-d063-4ece-9742-7b553b5391e2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:07.140957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-539e5e36-97a9-4e41-82cb-ba1f3fe44e33 None None] MFA Rules not processed for user `77893616533b4ad48dd6ba9dc6f5aee2`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:07.142179 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-539e5e36-97a9-4e41-82cb-ba1f3fe44e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:07.155227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:07.208069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84356ebd-7373-4940-8dc9-cc0f999e566e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:07.209725 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84356ebd-7373-4940-8dc9-cc0f999e566e service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:07.210533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84356ebd-7373-4940-8dc9-cc0f999e566e service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:07.210819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84356ebd-7373-4940-8dc9-cc0f999e566e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:07.213890 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-84356ebd-7373-4940-8dc9-cc0f999e566e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:07.249103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1038/2083] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:07.250859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e5e36-97a9-4e41-82cb-ba1f3fe44e33 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:07.308542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84356ebd-7373-4940-8dc9-cc0f999e566e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:07.310124 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84356ebd-7373-4940-8dc9-cc0f999e566e service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:07.321377 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1046/2084] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:07.335293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e5e36-97a9-4e41-82cb-ba1f3fe44e33 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:07.341746 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc7676-abff-4f94-a2ce-edafcb271c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:07.342485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc7676-abff-4f94-a2ce-edafcb271c18 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:07.342888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc7676-abff-4f94-a2ce-edafcb271c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:07.343114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc7676-abff-4f94-a2ce-edafcb271c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:07.343823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc7676-abff-4f94-a2ce-edafcb271c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:07.345821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc7676-abff-4f94-a2ce-edafcb271c18 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:07.389277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1039/2085] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:07 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:09:07.394047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84356ebd-7373-4940-8dc9-cc0f999e566e service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:07.407174 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84356ebd-7373-4940-8dc9-cc0f999e566e service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:07.505016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b383d31-1d7f-4c6f-8290-2512d0513434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:07.505755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b383d31-1d7f-4c6f-8290-2512d0513434 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:07.505908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b383d31-1d7f-4c6f-8290-2512d0513434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:07.506176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b383d31-1d7f-4c6f-8290-2512d0513434 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:07.507075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b383d31-1d7f-4c6f-8290-2512d0513434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:07.511535 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b383d31-1d7f-4c6f-8290-2512d0513434 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:07.538794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:07.553472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1047/2086] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:07 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:07.553809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc7676-abff-4f94-a2ce-edafcb271c18 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:07.567946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc7676-abff-4f94-a2ce-edafcb271c18 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:07.574501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e77bce-faab-4967-822e-32278796abb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:07.575726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e77bce-faab-4967-822e-32278796abb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:07.576062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e77bce-faab-4967-822e-32278796abb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:07.576348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e77bce-faab-4967-822e-32278796abb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:07.577472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e77bce-faab-4967-822e-32278796abb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:07.579108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e77bce-faab-4967-822e-32278796abb4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:07.591830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1040/2087] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:07.598622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b383d31-1d7f-4c6f-8290-2512d0513434 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:07.623557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b383d31-1d7f-4c6f-8290-2512d0513434 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:07.645112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b65b0-8ae5-487d-92fd-5ea1d1c8a841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:07.647817 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b65b0-8ae5-487d-92fd-5ea1d1c8a841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:07.648532 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b65b0-8ae5-487d-92fd-5ea1d1c8a841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:07.649100 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b65b0-8ae5-487d-92fd-5ea1d1c8a841 admin admin] PATH_INFO: `/v3/projects/4ffefa2621144bd083761fff56215d1a/users/e4e12d7392604806825265b580001e35/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:07.685804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b65b0-8ae5-487d-92fd-5ea1d1c8a841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ffefa2621144bd083761fff56215d1a, user_id=e4e12d7392604806825265b580001e35, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:07.692899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b65b0-8ae5-487d-92fd-5ea1d1c8a841 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:07.711160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1048/2088] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:07 2022] PUT /identity/v3/projects/4ffefa2621144bd083761fff56215d1a/users/e4e12d7392604806825265b580001e35/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:07.715618 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e77bce-faab-4967-822e-32278796abb4 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:07.730068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e77bce-faab-4967-822e-32278796abb4 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:07.800807 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb456f-1d6f-4733-862a-96019ff0e615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:07.802073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb456f-1d6f-4733-862a-96019ff0e615 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:07.802073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb456f-1d6f-4733-862a-96019ff0e615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:07.802073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb456f-1d6f-4733-862a-96019ff0e615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:07.802719 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb456f-1d6f-4733-862a-96019ff0e615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:07.804647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb456f-1d6f-4733-862a-96019ff0e615 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:07.818106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1041/2089] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:07.822764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b65b0-8ae5-487d-92fd-5ea1d1c8a841 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:07.837132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b65b0-8ae5-487d-92fd-5ea1d1c8a841 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:07.842382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434986b-7977-45fd-971b-d8f509f4a77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:07.843289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434986b-7977-45fd-971b-d8f509f4a77f admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:07.843608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434986b-7977-45fd-971b-d8f509f4a77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:07.843889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434986b-7977-45fd-971b-d8f509f4a77f admin admin] PATH_INFO: `/v3/projects/4ffefa2621144bd083761fff56215d1a/users/e4e12d7392604806825265b580001e35/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:07.846128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434986b-7977-45fd-971b-d8f509f4a77f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ffefa2621144bd083761fff56215d1a, user_id=e4e12d7392604806825265b580001e35, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:07.848860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434986b-7977-45fd-971b-d8f509f4a77f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:07.862202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1049/2090] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:07 2022] PUT /identity/v3/projects/4ffefa2621144bd083761fff56215d1a/users/e4e12d7392604806825265b580001e35/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:07.874418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441e2a9-3b5a-48c0-a305-5709cacc88a4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:07.874893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441e2a9-3b5a-48c0-a305-5709cacc88a4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:07.875182 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441e2a9-3b5a-48c0-a305-5709cacc88a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:07.906500 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a441e2a9-3b5a-48c0-a305-5709cacc88a4 None None] MFA Rules not processed for user `e4e12d7392604806825265b580001e35`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:07.907731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a441e2a9-3b5a-48c0-a305-5709cacc88a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:07.918723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:08.027812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1042/2091] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:08.039573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434986b-7977-45fd-971b-d8f509f4a77f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:08.110643 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434986b-7977-45fd-971b-d8f509f4a77f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:08.115018 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf9143-fcee-48ed-a4fa-8bbae3778705 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:08.115718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf9143-fcee-48ed-a4fa-8bbae3778705 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:08.116005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf9143-fcee-48ed-a4fa-8bbae3778705 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:08.116291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf9143-fcee-48ed-a4fa-8bbae3778705 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:08.136302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf9143-fcee-48ed-a4fa-8bbae3778705 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:08.146933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf9143-fcee-48ed-a4fa-8bbae3778705 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:08.164383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1050/2092] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:08.293431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441e2a9-3b5a-48c0-a305-5709cacc88a4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:08.322426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441e2a9-3b5a-48c0-a305-5709cacc88a4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:08.333395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fce68-ab03-41ae-afa6-0bfd6bf0c6d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:08.334196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fce68-ab03-41ae-afa6-0bfd6bf0c6d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:08.334561 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fce68-ab03-41ae-afa6-0bfd6bf0c6d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:08.334918 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fce68-ab03-41ae-afa6-0bfd6bf0c6d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:08.409503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fce68-ab03-41ae-afa6-0bfd6bf0c6d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:08.410966 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fce68-ab03-41ae-afa6-0bfd6bf0c6d9 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:08.421538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1043/2093] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:08.714381 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf9143-fcee-48ed-a4fa-8bbae3778705 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:08.722225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-21cf9143-fcee-48ed-a4fa-8bbae3778705 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:08.822519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf9143-fcee-48ed-a4fa-8bbae3778705 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:08.828396 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:08.829576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:08.829576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:08.830329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] PATH_INFO: `/v3/users/b283bdb8b4e94cdc9bd3e0212f09bce6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:08.832793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b283bdb8b4e94cdc9bd3e0212f09bce6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:08.834953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:08.926406 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b283bdb8b4e94cdc9bd3e0212f09bce6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:08.938164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b283bdb8b4e94cdc9bd3e0212f09bce6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:08.938914 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b283bdb8b4e94cdc9bd3e0212f09bce6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:08.953091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b283bdb8b4e94cdc9bd3e0212f09bce6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:08.954113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b283bdb8b4e94cdc9bd3e0212f09bce6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:08.956877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:08.986959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1051/2094] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:08 2022] DELETE /identity/v3/users/b283bdb8b4e94cdc9bd3e0212f09bce6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:08.991568 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ef28c-c8dd-4119-8dda-68d5a288f95d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:08.992153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ef28c-c8dd-4119-8dda-68d5a288f95d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:08.992602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ef28c-c8dd-4119-8dda-68d5a288f95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:09.012464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5c0ef28c-c8dd-4119-8dda-68d5a288f95d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:09.013908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5c0ef28c-c8dd-4119-8dda-68d5a288f95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:09.110972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1044/2095] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:09.123823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:09.125396 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:09.215816 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920f81b-50c7-4535-a515-4389778d5976 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:09.219969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7da715-c5da-4f00-9f2f-156cd63e464b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:09.220803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7da715-c5da-4f00-9f2f-156cd63e464b service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:09.221627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7da715-c5da-4f00-9f2f-156cd63e464b service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:09.221910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7da715-c5da-4f00-9f2f-156cd63e464b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:09.232445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7da715-c5da-4f00-9f2f-156cd63e464b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:09.233927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7da715-c5da-4f00-9f2f-156cd63e464b service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:09.246793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1052/2096] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:09.672454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ef28c-c8dd-4119-8dda-68d5a288f95d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:09.675748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5c0ef28c-c8dd-4119-8dda-68d5a288f95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:09.711968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ef28c-c8dd-4119-8dda-68d5a288f95d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:09.718871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:09.720013 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:09.720559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:09.720916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] PATH_INFO: `/v3/projects/8db2c4fd8b9840a2bf21107dbb291266` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:09.722611 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db2c4fd8b9840a2bf21107dbb291266)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:09.724912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:09.791969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:09.803786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db2c4fd8b9840a2bf21107dbb291266', 'request_id': 'req-696b1504-ec46-41fb-be2d-7007e79f207d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:09.824125 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db2c4fd8b9840a2bf21107dbb291266', 'request_id': 'req-696b1504-ec46-41fb-be2d-7007e79f207d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:09.839099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db2c4fd8b9840a2bf21107dbb291266', 'request_id': 'req-696b1504-ec46-41fb-be2d-7007e79f207d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:09.882088 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] The token cache is being invalidate because project 8db2c4fd8b9840a2bf21107dbb291266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:09.882513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:09.883745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:09.895523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1045/2097] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:09 2022] DELETE /identity/v3/projects/8db2c4fd8b9840a2bf21107dbb291266 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:11.674439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7da715-c5da-4f00-9f2f-156cd63e464b service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:11.676038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6e7da715-c5da-4f00-9f2f-156cd63e464b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:11.690822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7da715-c5da-4f00-9f2f-156cd63e464b service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:11.718146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:11.722138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:11.767774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243bfb8-b0b2-44a1-a0ad-ed2df1023757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:11.768851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243bfb8-b0b2-44a1-a0ad-ed2df1023757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:11.769139 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243bfb8-b0b2-44a1-a0ad-ed2df1023757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:11.769420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243bfb8-b0b2-44a1-a0ad-ed2df1023757 admin admin] PATH_INFO: `/v3/users/77893616533b4ad48dd6ba9dc6f5aee2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:11.770724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243bfb8-b0b2-44a1-a0ad-ed2df1023757 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77893616533b4ad48dd6ba9dc6f5aee2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:11.772698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243bfb8-b0b2-44a1-a0ad-ed2df1023757 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:11.797742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b1504-ec46-41fb-be2d-7007e79f207d admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:11.803235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7554f-506f-4aaf-a85b-4a7046589aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:11.804498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7554f-506f-4aaf-a85b-4a7046589aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:11.804794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7554f-506f-4aaf-a85b-4a7046589aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:11.805600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7554f-506f-4aaf-a85b-4a7046589aa2 admin admin] PATH_INFO: `/v3/users/d2624265a4b945099d024e826cf00123` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:11.807005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7554f-506f-4aaf-a85b-4a7046589aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2624265a4b945099d024e826cf00123)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:11.809069 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7554f-506f-4aaf-a85b-4a7046589aa2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:11.844496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2243bfb8-b0b2-44a1-a0ad-ed2df1023757 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77893616533b4ad48dd6ba9dc6f5aee2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:11.860346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2243bfb8-b0b2-44a1-a0ad-ed2df1023757 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77893616533b4ad48dd6ba9dc6f5aee2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:11.862630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2243bfb8-b0b2-44a1-a0ad-ed2df1023757 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77893616533b4ad48dd6ba9dc6f5aee2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:11.876245 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2243bfb8-b0b2-44a1-a0ad-ed2df1023757 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77893616533b4ad48dd6ba9dc6f5aee2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:11.877680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2243bfb8-b0b2-44a1-a0ad-ed2df1023757 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77893616533b4ad48dd6ba9dc6f5aee2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:11.877820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3bd7554f-506f-4aaf-a85b-4a7046589aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2624265a4b945099d024e826cf00123'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:11.878825 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3bd7554f-506f-4aaf-a85b-4a7046589aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2624265a4b945099d024e826cf00123'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:11.894123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3bd7554f-506f-4aaf-a85b-4a7046589aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2624265a4b945099d024e826cf00123'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:11.907311 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3bd7554f-506f-4aaf-a85b-4a7046589aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2624265a4b945099d024e826cf00123'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:11.908602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3bd7554f-506f-4aaf-a85b-4a7046589aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2624265a4b945099d024e826cf00123'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:11.912426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1053/2098] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:11 2022] DELETE /identity/v3/users/77893616533b4ad48dd6ba9dc6f5aee2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:11.919482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dccccf-0159-40bf-84ab-9cae68c9db1f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:11.919844 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dccccf-0159-40bf-84ab-9cae68c9db1f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:11.920233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dccccf-0159-40bf-84ab-9cae68c9db1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:11.931813 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:11.941300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1046/2099] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:11 2022] DELETE /identity/v3/users/d2624265a4b945099d024e826cf00123 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:11.943903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a01e8-2a76-4663-b5bc-21a2e28ab1c7 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:11.944417 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a01e8-2a76-4663-b5bc-21a2e28ab1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:11.944826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a01e8-2a76-4663-b5bc-21a2e28ab1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:11.957996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-61dccccf-0159-40bf-84ab-9cae68c9db1f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:11.961875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-61dccccf-0159-40bf-84ab-9cae68c9db1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:11.969657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6b0a01e8-2a76-4663-b5bc-21a2e28ab1c7 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:11.970950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6b0a01e8-2a76-4663-b5bc-21a2e28ab1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:12.029855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:12.050794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1047/2100] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:12.061610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a01e8-2a76-4663-b5bc-21a2e28ab1c7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:12.063026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6b0a01e8-2a76-4663-b5bc-21a2e28ab1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:12.063271 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1054/2101] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:12.072865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dccccf-0159-40bf-84ab-9cae68c9db1f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:12.075096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-61dccccf-0159-40bf-84ab-9cae68c9db1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:12.128089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a01e8-2a76-4663-b5bc-21a2e28ab1c7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:12.132412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3e62f-0869-47d3-bb62-2bc4b6a79d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:12.133113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3e62f-0869-47d3-bb62-2bc4b6a79d33 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:12.133398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3e62f-0869-47d3-bb62-2bc4b6a79d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:12.133687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3e62f-0869-47d3-bb62-2bc4b6a79d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:12.145281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3e62f-0869-47d3-bb62-2bc4b6a79d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:12.146799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dccccf-0159-40bf-84ab-9cae68c9db1f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:12.146942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3e62f-0869-47d3-bb62-2bc4b6a79d33 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:12.153713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199756c6-b416-46f7-bc3f-63ac8e73ce8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:12.153713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199756c6-b416-46f7-bc3f-63ac8e73ce8e service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:12.153713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199756c6-b416-46f7-bc3f-63ac8e73ce8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:12.154512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199756c6-b416-46f7-bc3f-63ac8e73ce8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:12.158485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1048/2102] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:12.165999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199756c6-b416-46f7-bc3f-63ac8e73ce8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:12.167512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199756c6-b416-46f7-bc3f-63ac8e73ce8e service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:12.178823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1055/2103] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:12.649401 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3e62f-0869-47d3-bb62-2bc4b6a79d33 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:12.651062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-afe3e62f-0869-47d3-bb62-2bc4b6a79d33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:12.655547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199756c6-b416-46f7-bc3f-63ac8e73ce8e service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:12.657349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-199756c6-b416-46f7-bc3f-63ac8e73ce8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:12.667251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3e62f-0869-47d3-bb62-2bc4b6a79d33 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:12.672101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:12.672984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:12.673272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:12.673554 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] PATH_INFO: `/v3/projects/6f547d682d364b2789dd11bb434a89ef` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:12.675275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f547d682d364b2789dd11bb434a89ef)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:12.676959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:12.681249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199756c6-b416-46f7-bc3f-63ac8e73ce8e service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:12.690877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:12.691961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:12.692322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:12.692651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] PATH_INFO: `/v3/projects/9518beef1490422b91771ceed94a995b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:12.694038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9518beef1490422b91771ceed94a995b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:12.696239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:12.735274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f547d682d364b2789dd11bb434a89ef', 'request_id': 'req-c7dbbc93-923e-4897-904f-f7c5ffda44b6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.749805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f547d682d364b2789dd11bb434a89ef', 'request_id': 'req-c7dbbc93-923e-4897-904f-f7c5ffda44b6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.756795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9518beef1490422b91771ceed94a995b', 'request_id': 'req-6a776ce8-9b76-4f35-ae35-dd51d6f53285'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.757061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f547d682d364b2789dd11bb434a89ef', 'request_id': 'req-c7dbbc93-923e-4897-904f-f7c5ffda44b6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.764633 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9518beef1490422b91771ceed94a995b', 'request_id': 'req-6a776ce8-9b76-4f35-ae35-dd51d6f53285'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.771235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9518beef1490422b91771ceed94a995b', 'request_id': 'req-6a776ce8-9b76-4f35-ae35-dd51d6f53285'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.777887 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] The token cache is being invalidate because project 6f547d682d364b2789dd11bb434a89ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:12.778333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.779247 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.781286 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1049/2104] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:12 2022] DELETE /identity/v3/projects/6f547d682d364b2789dd11bb434a89ef => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:12.785105 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:12.786600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:12.795079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] The token cache is being invalidate because project 9518beef1490422b91771ceed94a995b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:12.795541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.796220 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.798282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1056/2105] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:12 2022] DELETE /identity/v3/projects/9518beef1490422b91771ceed94a995b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:12.801722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:12.803082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:12.841853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:12.861254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbbc93-923e-4897-904f-f7c5ffda44b6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:12.885422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a776ce8-9b76-4f35-ae35-dd51d6f53285 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:12.886412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:12.892492 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:12.893389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:12.893835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:12.894125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] PATH_INFO: `/v3/users/e4e12d7392604806825265b580001e35` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:12.895460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e12d7392604806825265b580001e35)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:12.897691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:12.937336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:12.938236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:12.938525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:12.938810 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] PATH_INFO: `/v3/users/7a57ce9dbf324046b486758fddfaa309` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:12.940061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a57ce9dbf324046b486758fddfaa309)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:12.942070 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:12.954598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e12d7392604806825265b580001e35'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.964600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e12d7392604806825265b580001e35'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.965388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e12d7392604806825265b580001e35'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.973977 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e12d7392604806825265b580001e35'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.974764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e12d7392604806825265b580001e35'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.994148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1057/2106] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:12 2022] DELETE /identity/v3/users/e4e12d7392604806825265b580001e35 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:12.997988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a57ce9dbf324046b486758fddfaa309'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:12.998983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a57ce9dbf324046b486758fddfaa309'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.009394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a57ce9dbf324046b486758fddfaa309'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.018581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a57ce9dbf324046b486758fddfaa309'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.019371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a57ce9dbf324046b486758fddfaa309'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.039802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1050/2107] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:12 2022] DELETE /identity/v3/users/7a57ce9dbf324046b486758fddfaa309 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:13.467042 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:13.468717 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:13.474872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:13.476425 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:13.564264 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587a9a6-a99a-489a-affd-d12ba1c84047 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:13.565168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507a352-12e4-4761-9f34-fa85c95c4ac6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:13.569210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:13.569731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:13.570074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:13.570362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:13.571145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:13.571310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] PATH_INFO: `/v3/projects/bb6f3f966dd342879dba1629382940bb` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:13.571455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:13.571729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13 admin admin] PATH_INFO: `/v3/projects/4ffefa2621144bd083761fff56215d1a` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:13.572681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb6f3f966dd342879dba1629382940bb)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:13.572942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ffefa2621144bd083761fff56215d1a)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:13.574738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:13.574846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:13.624027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:13.627817 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb6f3f966dd342879dba1629382940bb', 'request_id': 'req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.628222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ffefa2621144bd083761fff56215d1a', 'request_id': 'req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.630021 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:13.636506 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb6f3f966dd342879dba1629382940bb', 'request_id': 'req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.641274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ffefa2621144bd083761fff56215d1a', 'request_id': 'req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.643426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb6f3f966dd342879dba1629382940bb', 'request_id': 'req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.649904 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ffefa2621144bd083761fff56215d1a', 'request_id': 'req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.662036 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] The token cache is being invalidate because project bb6f3f966dd342879dba1629382940bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:13.662328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.663038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.665649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13 admin admin] The token cache is being invalidate because project 4ffefa2621144bd083761fff56215d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:13.666158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.666892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9ecedb7b-40f4-44e0-9c32-a8c3f56b5d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.667865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1058/2108] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:13 2022] DELETE /identity/v3/projects/bb6f3f966dd342879dba1629382940bb => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:13.671846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1051/2109] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:13 2022] DELETE /identity/v3/projects/4ffefa2621144bd083761fff56215d1a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:13.672007 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:13.673210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:13.673430 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cc082-dc5a-4a6a-a813-c044039822e0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:13.674009 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cc082-dc5a-4a6a-a813-c044039822e0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:13.674305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cc082-dc5a-4a6a-a813-c044039822e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:13.693787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b77cc082-dc5a-4a6a-a813-c044039822e0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:13.695016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b77cc082-dc5a-4a6a-a813-c044039822e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:13.737736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfeeaf-fef3-4c2d-91f7-afb5511ee5fb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:13.742478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:13.743353 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:13.743670 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:13.743950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] PATH_INFO: `/v3/users/a8ece0820a684704a3c01d0f5af63069` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:13.745150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8ece0820a684704a3c01d0f5af63069)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:13.747134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:13.756798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1052/2110] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:13.764980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e3789-09f7-47c9-8db5-5ec2d37b4adc None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:13.765273 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e3789-09f7-47c9-8db5-5ec2d37b4adc None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:13.765558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e3789-09f7-47c9-8db5-5ec2d37b4adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:13.784664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-fc9e3789-09f7-47c9-8db5-5ec2d37b4adc None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:13.785908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fc9e3789-09f7-47c9-8db5-5ec2d37b4adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:13.794147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1053/2111] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:13.798836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e3789-09f7-47c9-8db5-5ec2d37b4adc None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:13.803264 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8ece0820a684704a3c01d0f5af63069'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.808668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e3789-09f7-47c9-8db5-5ec2d37b4adc None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:13.812278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8ece0820a684704a3c01d0f5af63069'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.813060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8ece0820a684704a3c01d0f5af63069'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.813801 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cd81c-2eb7-4206-bfdd-5a6a5639a61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:13.814481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cd81c-2eb7-4206-bfdd-5a6a5639a61a admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:13.814770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cd81c-2eb7-4206-bfdd-5a6a5639a61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:13.815057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cd81c-2eb7-4206-bfdd-5a6a5639a61a admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:13.815837 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cd81c-2eb7-4206-bfdd-5a6a5639a61a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:09:13.816180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cd81c-2eb7-4206-bfdd-5a6a5639a61a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:13.817785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cd81c-2eb7-4206-bfdd-5a6a5639a61a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:13.823697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8ece0820a684704a3c01d0f5af63069'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.824712 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8ece0820a684704a3c01d0f5af63069'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:13.831368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1054/2112] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:09:13.836264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cd81c-2eb7-4206-bfdd-5a6a5639a61a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:13.838695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-585cd81c-2eb7-4206-bfdd-5a6a5639a61a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:13.847474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1059/2113] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:13 2022] DELETE /identity/v3/users/a8ece0820a684704a3c01d0f5af63069 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:13.858234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cd81c-2eb7-4206-bfdd-5a6a5639a61a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:13.950147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4a2fa-599e-40f1-a686-1c6b35bff3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:13.951262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4a2fa-599e-40f1-a686-1c6b35bff3bb admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:13.951780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4a2fa-599e-40f1-a686-1c6b35bff3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:13.952205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4a2fa-599e-40f1-a686-1c6b35bff3bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:13.953204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4a2fa-599e-40f1-a686-1c6b35bff3bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:13.955954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4a2fa-599e-40f1-a686-1c6b35bff3bb admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.007365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1055/2114] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:13 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:09:14.009177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.019659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40678a7e-acbc-4bf8-9895-e57dab3ac3bb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.085374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670aaf01-1715-47a9-b55d-f88dc873c9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.086763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aaf01-1715-47a9-b55d-f88dc873c9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.086987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aaf01-1715-47a9-b55d-f88dc873c9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.087338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aaf01-1715-47a9-b55d-f88dc873c9d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.088023 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670aaf01-1715-47a9-b55d-f88dc873c9d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.090212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670aaf01-1715-47a9-b55d-f88dc873c9d1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.099468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:14.122628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1060/2115] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:14 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:14.126324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4a2fa-599e-40f1-a686-1c6b35bff3bb admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.143707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4a2fa-599e-40f1-a686-1c6b35bff3bb admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.152281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acc549-9628-4c8b-ad72-ee91393585e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.153328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acc549-9628-4c8b-ad72-ee91393585e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.153684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acc549-9628-4c8b-ad72-ee91393585e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.153970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acc549-9628-4c8b-ad72-ee91393585e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.154973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acc549-9628-4c8b-ad72-ee91393585e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.156371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acc549-9628-4c8b-ad72-ee91393585e1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.169303 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1056/2116] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:14.173468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670aaf01-1715-47a9-b55d-f88dc873c9d1 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.184900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670aaf01-1715-47a9-b55d-f88dc873c9d1 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.189824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3cb90-e25b-4147-80a1-f64d4339cc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.190508 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3cb90-e25b-4147-80a1-f64d4339cc40 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.190785 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3cb90-e25b-4147-80a1-f64d4339cc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.191944 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3cb90-e25b-4147-80a1-f64d4339cc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.191944 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3cb90-e25b-4147-80a1-f64d4339cc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.193449 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3cb90-e25b-4147-80a1-f64d4339cc40 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.206739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1061/2117] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:14.210800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acc549-9628-4c8b-ad72-ee91393585e1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.226861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acc549-9628-4c8b-ad72-ee91393585e1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.233828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e188d14-3fce-44bc-8be8-60810ce0e5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.235353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e188d14-3fce-44bc-8be8-60810ce0e5c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.235892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e188d14-3fce-44bc-8be8-60810ce0e5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.236488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e188d14-3fce-44bc-8be8-60810ce0e5c6 admin admin] PATH_INFO: `/v3/projects/f54cf5139f98409e9653ce4a8ab49c1b/users/e26f9f2a46204919891d36926c075365/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.246272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3cb90-e25b-4147-80a1-f64d4339cc40 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.248366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b3a3cb90-e25b-4147-80a1-f64d4339cc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:14.267573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3cb90-e25b-4147-80a1-f64d4339cc40 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.267783 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e188d14-3fce-44bc-8be8-60810ce0e5c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54cf5139f98409e9653ce4a8ab49c1b, user_id=e26f9f2a46204919891d36926c075365, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.270718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e188d14-3fce-44bc-8be8-60810ce0e5c6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.272989 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460c078-b55e-4f80-9188-567c48fc3faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.274613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460c078-b55e-4f80-9188-567c48fc3faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.275052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460c078-b55e-4f80-9188-567c48fc3faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.275173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460c078-b55e-4f80-9188-567c48fc3faa admin admin] PATH_INFO: `/v3/projects/c90ba2f05e444c9c93925b20ee9c9a2c` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.276528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460c078-b55e-4f80-9188-567c48fc3faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c90ba2f05e444c9c93925b20ee9c9a2c)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.279053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460c078-b55e-4f80-9188-567c48fc3faa admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.288313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1057/2118] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:14 2022] PUT /identity/v3/projects/f54cf5139f98409e9653ce4a8ab49c1b/users/e26f9f2a46204919891d36926c075365/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:14.299304 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b0543-97f7-4540-99c7-868b5185ec36 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.299752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b0543-97f7-4540-99c7-868b5185ec36 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.299929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b0543-97f7-4540-99c7-868b5185ec36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.306635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:14.333900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-785b0543-97f7-4540-99c7-868b5185ec36 None None] MFA Rules not processed for user `e26f9f2a46204919891d36926c075365`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:14.335183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-785b0543-97f7-4540-99c7-868b5185ec36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:14.347931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8460c078-b55e-4f80-9188-567c48fc3faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c90ba2f05e444c9c93925b20ee9c9a2c', 'request_id': 'req-8460c078-b55e-4f80-9188-567c48fc3faa'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:14.356646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8460c078-b55e-4f80-9188-567c48fc3faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c90ba2f05e444c9c93925b20ee9c9a2c', 'request_id': 'req-8460c078-b55e-4f80-9188-567c48fc3faa'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:14.363205 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8460c078-b55e-4f80-9188-567c48fc3faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c90ba2f05e444c9c93925b20ee9c9a2c', 'request_id': 'req-8460c078-b55e-4f80-9188-567c48fc3faa'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:14.378459 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8460c078-b55e-4f80-9188-567c48fc3faa admin admin] The token cache is being invalidate because project c90ba2f05e444c9c93925b20ee9c9a2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:14.378757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8460c078-b55e-4f80-9188-567c48fc3faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:14.379476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8460c078-b55e-4f80-9188-567c48fc3faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:14.381073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1062/2119] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:14 2022] DELETE /identity/v3/projects/c90ba2f05e444c9c93925b20ee9c9a2c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:14.386457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ceee2-a286-4461-876b-1833acf7ac58 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.386752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ceee2-a286-4461-876b-1833acf7ac58 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.387039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ceee2-a286-4461-876b-1833acf7ac58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.406661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c53ceee2-a286-4461-876b-1833acf7ac58 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:14.407851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c53ceee2-a286-4461-876b-1833acf7ac58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:14.422893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1058/2120] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:14.430928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b0543-97f7-4540-99c7-868b5185ec36 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.432428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-785b0543-97f7-4540-99c7-868b5185ec36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:14.501565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1063/2121] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:14.516459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b0543-97f7-4540-99c7-868b5185ec36 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.516630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928e6fd-ae8c-4fdb-a40a-6544b984e0a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.517083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e63aff-f584-40de-a445-70954f058f55 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.517294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928e6fd-ae8c-4fdb-a40a-6544b984e0a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.517378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e63aff-f584-40de-a445-70954f058f55 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.517595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928e6fd-ae8c-4fdb-a40a-6544b984e0a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.517682 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e63aff-f584-40de-a445-70954f058f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.517860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928e6fd-ae8c-4fdb-a40a-6544b984e0a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.529073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928e6fd-ae8c-4fdb-a40a-6544b984e0a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.530657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928e6fd-ae8c-4fdb-a40a-6544b984e0a7 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.543370 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1059/2122] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:14.545148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-34e63aff-f584-40de-a445-70954f058f55 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:14.548257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-34e63aff-f584-40de-a445-70954f058f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:14.559424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1064/2123] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:14.561970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928e6fd-ae8c-4fdb-a40a-6544b984e0a7 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.564929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e63aff-f584-40de-a445-70954f058f55 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.566598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-34e63aff-f584-40de-a445-70954f058f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:14.575695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928e6fd-ae8c-4fdb-a40a-6544b984e0a7 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.580460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e63aff-f584-40de-a445-70954f058f55 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.580690 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d34695-4e90-45f9-a9b3-d86d87a14783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.581651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d34695-4e90-45f9-a9b3-d86d87a14783 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.581651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d34695-4e90-45f9-a9b3-d86d87a14783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.581937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d34695-4e90-45f9-a9b3-d86d87a14783 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.583004 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d34695-4e90-45f9-a9b3-d86d87a14783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:09:14.583144 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d34695-4e90-45f9-a9b3-d86d87a14783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.584734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d34695-4e90-45f9-a9b3-d86d87a14783 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.587028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f583918-38be-4027-891b-46730bc54911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.588069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f583918-38be-4027-891b-46730bc54911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.588452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f583918-38be-4027-891b-46730bc54911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.588821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f583918-38be-4027-891b-46730bc54911 admin admin] PATH_INFO: `/v3/users/e26f9f2a46204919891d36926c075365` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.590514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f583918-38be-4027-891b-46730bc54911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e26f9f2a46204919891d36926c075365)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.592927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f583918-38be-4027-891b-46730bc54911 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.599033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1060/2124] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:09:14.604625 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d34695-4e90-45f9-a9b3-d86d87a14783 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.615983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d34695-4e90-45f9-a9b3-d86d87a14783 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.622345 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99eead4-2440-4d6b-927e-56f476887e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.623292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99eead4-2440-4d6b-927e-56f476887e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.623712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99eead4-2440-4d6b-927e-56f476887e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.624175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99eead4-2440-4d6b-927e-56f476887e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.625147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99eead4-2440-4d6b-927e-56f476887e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.627484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99eead4-2440-4d6b-927e-56f476887e35 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.655990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f583918-38be-4027-891b-46730bc54911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e26f9f2a46204919891d36926c075365'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:14.661477 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1061/2125] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:14 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:14.661643 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=5,keystone:SELECT=81,keystone:DELETE=14 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:14.663423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99eead4-2440-4d6b-927e-56f476887e35 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.669320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f583918-38be-4027-891b-46730bc54911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e26f9f2a46204919891d36926c075365'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:14.670064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f583918-38be-4027-891b-46730bc54911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e26f9f2a46204919891d36926c075365'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:14.678785 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f583918-38be-4027-891b-46730bc54911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e26f9f2a46204919891d36926c075365'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:14.679790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f583918-38be-4027-891b-46730bc54911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e26f9f2a46204919891d36926c075365'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:14.701837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1065/2126] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:14 2022] DELETE /identity/v3/users/e26f9f2a46204919891d36926c075365 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:14.706228 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf0e0d-e57c-46ad-a2b5-8da65b05aab6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.706726 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf0e0d-e57c-46ad-a2b5-8da65b05aab6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.707012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf0e0d-e57c-46ad-a2b5-8da65b05aab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.726476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2acf0e0d-e57c-46ad-a2b5-8da65b05aab6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:14.728933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2acf0e0d-e57c-46ad-a2b5-8da65b05aab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:14.736049 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99eead4-2440-4d6b-927e-56f476887e35 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.740497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:14.805898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1066/2127] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:14.809463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1736a83-f2a0-403a-9596-11077eb1a489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.810179 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1736a83-f2a0-403a-9596-11077eb1a489 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.810469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1736a83-f2a0-403a-9596-11077eb1a489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.810759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1736a83-f2a0-403a-9596-11077eb1a489 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.812115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1736a83-f2a0-403a-9596-11077eb1a489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.813524 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1736a83-f2a0-403a-9596-11077eb1a489 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.817496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf0e0d-e57c-46ad-a2b5-8da65b05aab6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.818958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2acf0e0d-e57c-46ad-a2b5-8da65b05aab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:14.849973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1062/2128] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:14 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:14.849973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1736a83-f2a0-403a-9596-11077eb1a489 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.852684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a1736a83-f2a0-403a-9596-11077eb1a489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:14.864299 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1736a83-f2a0-403a-9596-11077eb1a489 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.870141 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dde65-62cf-4fbd-8418-3b2d38c3a224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.871060 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dde65-62cf-4fbd-8418-3b2d38c3a224 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.871343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dde65-62cf-4fbd-8418-3b2d38c3a224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.871671 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dde65-62cf-4fbd-8418-3b2d38c3a224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.872461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dde65-62cf-4fbd-8418-3b2d38c3a224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.874100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dde65-62cf-4fbd-8418-3b2d38c3a224 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.885752 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1063/2129] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:14.889785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dde65-62cf-4fbd-8418-3b2d38c3a224 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.901841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dde65-62cf-4fbd-8418-3b2d38c3a224 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.906597 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d284e05-556e-462c-bbb4-4587c5a15433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.907278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d284e05-556e-462c-bbb4-4587c5a15433 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.907602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d284e05-556e-462c-bbb4-4587c5a15433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.907889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d284e05-556e-462c-bbb4-4587c5a15433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.908633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d284e05-556e-462c-bbb4-4587c5a15433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.911337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d284e05-556e-462c-bbb4-4587c5a15433 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.912665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf0e0d-e57c-46ad-a2b5-8da65b05aab6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.917721 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba51313-b63c-47a9-91e4-f99336975a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.918777 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba51313-b63c-47a9-91e4-f99336975a43 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.919599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba51313-b63c-47a9-91e4-f99336975a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.919599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba51313-b63c-47a9-91e4-f99336975a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.922548 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1064/2130] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:09:14.926711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d284e05-556e-462c-bbb4-4587c5a15433 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:14.932718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba51313-b63c-47a9-91e4-f99336975a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.934292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba51313-b63c-47a9-91e4-f99336975a43 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.938808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d284e05-556e-462c-bbb4-4587c5a15433 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:14.944510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c21b2-2a83-4772-8850-e3276a1e9f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:14.944670 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1067/2131] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:14.946344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c21b2-2a83-4772-8850-e3276a1e9f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.946953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c21b2-2a83-4772-8850-e3276a1e9f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.947193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c21b2-2a83-4772-8850-e3276a1e9f36 admin admin] PATH_INFO: `/v3/projects/5ce5cf1763614953b1d376be9e77f2fd/users/1f6f5ea786e743b69b9311e49af302d6/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:14.964165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c21b2-2a83-4772-8850-e3276a1e9f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ce5cf1763614953b1d376be9e77f2fd, user_id=1f6f5ea786e743b69b9311e49af302d6, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:14.966956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c21b2-2a83-4772-8850-e3276a1e9f36 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:14.981337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1065/2132] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:14 2022] PUT /identity/v3/projects/5ce5cf1763614953b1d376be9e77f2fd/users/1f6f5ea786e743b69b9311e49af302d6/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:14.991296 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd557cf-d438-42bb-a92b-3f1fd4ed0e13 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:14.991627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd557cf-d438-42bb-a92b-3f1fd4ed0e13 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:14.991936 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd557cf-d438-42bb-a92b-3f1fd4ed0e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:15.021026 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9dd557cf-d438-42bb-a92b-3f1fd4ed0e13 None None] MFA Rules not processed for user `1f6f5ea786e743b69b9311e49af302d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:15.022500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9dd557cf-d438-42bb-a92b-3f1fd4ed0e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:15.088426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:15.092824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1068/2133] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:15.110474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c21b2-2a83-4772-8850-e3276a1e9f36 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:15.112572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9a2c21b2-2a83-4772-8850-e3276a1e9f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:15.175208 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c21b2-2a83-4772-8850-e3276a1e9f36 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:15.179982 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750d303-c3ee-4458-93f6-158508448a25 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1ad1ffe4d99420cb568d6154f779838', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:15.181158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750d303-c3ee-4458-93f6-158508448a25 service glance] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:15.181453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750d303-c3ee-4458-93f6-158508448a25 service glance] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:15.181738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750d303-c3ee-4458-93f6-158508448a25 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:15.194221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750d303-c3ee-4458-93f6-158508448a25 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:15.195810 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750d303-c3ee-4458-93f6-158508448a25 service glance] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:15.207759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1066/2134] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:15.287543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd557cf-d438-42bb-a92b-3f1fd4ed0e13 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:15.289657 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9dd557cf-d438-42bb-a92b-3f1fd4ed0e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:15.402746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd557cf-d438-42bb-a92b-3f1fd4ed0e13 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:15.410032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0e31e-fd64-4a6d-ac78-d065adb48061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:15.411236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0e31e-fd64-4a6d-ac78-d065adb48061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:15.411608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0e31e-fd64-4a6d-ac78-d065adb48061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:15.411904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0e31e-fd64-4a6d-ac78-d065adb48061 admin admin] PATH_INFO: `/v3/projects/f54cf5139f98409e9653ce4a8ab49c1b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:15.413550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0e31e-fd64-4a6d-ac78-d065adb48061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f54cf5139f98409e9653ce4a8ab49c1b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:15.416057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0e31e-fd64-4a6d-ac78-d065adb48061 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:15.468149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fed0e31e-fd64-4a6d-ac78-d065adb48061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f54cf5139f98409e9653ce4a8ab49c1b', 'request_id': 'req-fed0e31e-fd64-4a6d-ac78-d065adb48061'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:15.474545 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fed0e31e-fd64-4a6d-ac78-d065adb48061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f54cf5139f98409e9653ce4a8ab49c1b', 'request_id': 'req-fed0e31e-fd64-4a6d-ac78-d065adb48061'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:15.480334 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fed0e31e-fd64-4a6d-ac78-d065adb48061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f54cf5139f98409e9653ce4a8ab49c1b', 'request_id': 'req-fed0e31e-fd64-4a6d-ac78-d065adb48061'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:15.496488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fed0e31e-fd64-4a6d-ac78-d065adb48061 admin admin] The token cache is being invalidate because project f54cf5139f98409e9653ce4a8ab49c1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:15.496790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fed0e31e-fd64-4a6d-ac78-d065adb48061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:15.497609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-fed0e31e-fd64-4a6d-ac78-d065adb48061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:15.500175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1069/2135] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:15 2022] DELETE /identity/v3/projects/f54cf5139f98409e9653ce4a8ab49c1b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:15.505157 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179df22-20a9-4633-b97e-c5bba75bfb80 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:15.505503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179df22-20a9-4633-b97e-c5bba75bfb80 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:15.505767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179df22-20a9-4633-b97e-c5bba75bfb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:15.528980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-1179df22-20a9-4633-b97e-c5bba75bfb80 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:15.530367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1179df22-20a9-4633-b97e-c5bba75bfb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:15.608265 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:15.627357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1067/2136] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:15.636296 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57cd8f-7208-4c3e-aff6-d3dd049ff821 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:15.636594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57cd8f-7208-4c3e-aff6-d3dd049ff821 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:15.636979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57cd8f-7208-4c3e-aff6-d3dd049ff821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:15.664072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fb57cd8f-7208-4c3e-aff6-d3dd049ff821 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:15.666025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fb57cd8f-7208-4c3e-aff6-d3dd049ff821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:15.678667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1070/2137] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:15.679986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179df22-20a9-4633-b97e-c5bba75bfb80 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:15.692324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179df22-20a9-4633-b97e-c5bba75bfb80 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:15.697206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3829-7062-4849-998d-a22acc2a263c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:15.697891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3829-7062-4849-998d-a22acc2a263c admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:15.698351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3829-7062-4849-998d-a22acc2a263c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:15.698649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3829-7062-4849-998d-a22acc2a263c admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:15.699416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c3829-7062-4849-998d-a22acc2a263c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:09:15.699735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c3829-7062-4849-998d-a22acc2a263c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:15.702489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c3829-7062-4849-998d-a22acc2a263c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:15.717583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1068/2138] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:09:15.721697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57cd8f-7208-4c3e-aff6-d3dd049ff821 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:15.731933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57cd8f-7208-4c3e-aff6-d3dd049ff821 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:15.736745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d244d-6e33-422c-adb1-e67b995a26ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:15.737469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d244d-6e33-422c-adb1-e67b995a26ad admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:15.737712 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d244d-6e33-422c-adb1-e67b995a26ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:15.737993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d244d-6e33-422c-adb1-e67b995a26ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:15.738698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d244d-6e33-422c-adb1-e67b995a26ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:15.740731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d244d-6e33-422c-adb1-e67b995a26ad admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:15.771440 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1071/2139] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:15 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:15.777533 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3829-7062-4849-998d-a22acc2a263c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:15.788675 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3829-7062-4849-998d-a22acc2a263c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:15.872114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec2f88-4e65-4e11-b086-243222d116ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:15.873777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec2f88-4e65-4e11-b086-243222d116ab admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:15.873777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec2f88-4e65-4e11-b086-243222d116ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:15.873777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec2f88-4e65-4e11-b086-243222d116ab admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:15.874420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec2f88-4e65-4e11-b086-243222d116ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:15.877212 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec2f88-4e65-4e11-b086-243222d116ab admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:15.902974 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1069/2140] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:15 2022] POST /identity/v3/users => generated 281 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:15.907194 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d244d-6e33-422c-adb1-e67b995a26ad admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:15.918864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d244d-6e33-422c-adb1-e67b995a26ad admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:15.924182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dcd06-7218-4a3e-918e-05814fee66c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:15.924951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dcd06-7218-4a3e-918e-05814fee66c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:15.925134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dcd06-7218-4a3e-918e-05814fee66c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:15.925445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dcd06-7218-4a3e-918e-05814fee66c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:15.926179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dcd06-7218-4a3e-918e-05814fee66c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:15.927822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dcd06-7218-4a3e-918e-05814fee66c9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:15.943730 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1072/2141] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:15.948359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec2f88-4e65-4e11-b086-243222d116ab admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:15.959203 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec2f88-4e65-4e11-b086-243222d116ab admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:15.964093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2644e0-8907-42bb-b870-a71ea8776e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:15.964889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2644e0-8907-42bb-b870-a71ea8776e53 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:15.965162 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2644e0-8907-42bb-b870-a71ea8776e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:15.965477 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2644e0-8907-42bb-b870-a71ea8776e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:15.966183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2644e0-8907-42bb-b870-a71ea8776e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:15.967842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2644e0-8907-42bb-b870-a71ea8776e53 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:15.978655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1070/2142] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:15.983107 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dcd06-7218-4a3e-918e-05814fee66c9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:15.994193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dcd06-7218-4a3e-918e-05814fee66c9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:15.999193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657792c8-b216-4f87-830f-4d7cc645e399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:16.000094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657792c8-b216-4f87-830f-4d7cc645e399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:16.000389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657792c8-b216-4f87-830f-4d7cc645e399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:16.000690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657792c8-b216-4f87-830f-4d7cc645e399 admin admin] PATH_INFO: `/v3/projects/e98f156cbabe4016b2e655561a146fce/users/0a76517487e9425888aec2a44c9086c6/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:16.015427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657792c8-b216-4f87-830f-4d7cc645e399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98f156cbabe4016b2e655561a146fce, user_id=0a76517487e9425888aec2a44c9086c6, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:16.018100 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657792c8-b216-4f87-830f-4d7cc645e399 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:16.031714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1073/2143] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:15 2022] PUT /identity/v3/projects/e98f156cbabe4016b2e655561a146fce/users/0a76517487e9425888aec2a44c9086c6/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:16.042402 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69484dc-e4d7-4a3d-9662-ea595b807dca None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:16.042696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69484dc-e4d7-4a3d-9662-ea595b807dca None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:16.042992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69484dc-e4d7-4a3d-9662-ea595b807dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:16.070440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b69484dc-e4d7-4a3d-9662-ea595b807dca None None] MFA Rules not processed for user `0a76517487e9425888aec2a44c9086c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:16.071697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b69484dc-e4d7-4a3d-9662-ea595b807dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:16.135230 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1071/2144] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:16.168821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657792c8-b216-4f87-830f-4d7cc645e399 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:16.170911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-657792c8-b216-4f87-830f-4d7cc645e399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:16.203482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:16.233666 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657792c8-b216-4f87-830f-4d7cc645e399 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:16.237183 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093ba38-1858-4440-910f-90d589dab211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:16.238028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093ba38-1858-4440-910f-90d589dab211 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:16.238122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093ba38-1858-4440-910f-90d589dab211 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:16.238386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093ba38-1858-4440-910f-90d589dab211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:16.247254 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093ba38-1858-4440-910f-90d589dab211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:16.248755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093ba38-1858-4440-910f-90d589dab211 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:16.258027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1074/2145] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:16.271893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69484dc-e4d7-4a3d-9662-ea595b807dca None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:16.321517 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:16.338194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69484dc-e4d7-4a3d-9662-ea595b807dca None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:16.342856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:16.343762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:16.344046 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:16.344324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] PATH_INFO: `/v3/users/0a76517487e9425888aec2a44c9086c6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:16.345567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a76517487e9425888aec2a44c9086c6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:16.347532 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:16.415581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a76517487e9425888aec2a44c9086c6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:16.417406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a76517487e9425888aec2a44c9086c6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:16.427276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a76517487e9425888aec2a44c9086c6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:16.444298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a76517487e9425888aec2a44c9086c6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:16.445797 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a76517487e9425888aec2a44c9086c6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:16.489210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1072/2146] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:16 2022] DELETE /identity/v3/users/0a76517487e9425888aec2a44c9086c6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:16.493054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5eaaa4-9e03-4df2-93f6-a94197af0d98 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:16.493404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5eaaa4-9e03-4df2-93f6-a94197af0d98 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:16.493749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5eaaa4-9e03-4df2-93f6-a94197af0d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:16.525383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9c5eaaa4-9e03-4df2-93f6-a94197af0d98 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:16.527249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9c5eaaa4-9e03-4df2-93f6-a94197af0d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:16.630995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1075/2147] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:16.640996 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:16.645047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:16.737626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacbc1d-c285-42e8-ab91-f45d6b9222f3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:16.742680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322f09a-532f-4ff0-9968-5183eb117461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:16.742680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322f09a-532f-4ff0-9968-5183eb117461 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:16.742680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322f09a-532f-4ff0-9968-5183eb117461 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:16.742680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322f09a-532f-4ff0-9968-5183eb117461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:16.753639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3322f09a-532f-4ff0-9968-5183eb117461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:16.755240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3322f09a-532f-4ff0-9968-5183eb117461 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:16.771779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1073/2148] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:17.234366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5eaaa4-9e03-4df2-93f6-a94197af0d98 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:17.236033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9c5eaaa4-9e03-4df2-93f6-a94197af0d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:17.252623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5eaaa4-9e03-4df2-93f6-a94197af0d98 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:17.257537 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:17.259006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.259285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.259590 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6 admin admin] PATH_INFO: `/v3/projects/e98f156cbabe4016b2e655561a146fce` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.260825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e98f156cbabe4016b2e655561a146fce)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:17.262865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:17.316087 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e98f156cbabe4016b2e655561a146fce', 'request_id': 'req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:17.322453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e98f156cbabe4016b2e655561a146fce', 'request_id': 'req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:17.329052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e98f156cbabe4016b2e655561a146fce', 'request_id': 'req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:17.341656 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6 admin admin] The token cache is being invalidate because project e98f156cbabe4016b2e655561a146fce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:17.341950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:17.342593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-039d2628-9ab5-4dd4-b2c7-c6fda2682dd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:17.344204 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1076/2149] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:17 2022] DELETE /identity/v3/projects/e98f156cbabe4016b2e655561a146fce => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:17.349096 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ab2e8-e5eb-4ff4-964f-4f0c305f9537 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.349601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ab2e8-e5eb-4ff4-964f-4f0c305f9537 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.349741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ab2e8-e5eb-4ff4-964f-4f0c305f9537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.372165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9a8ab2e8-e5eb-4ff4-964f-4f0c305f9537 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:17.373530 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9a8ab2e8-e5eb-4ff4-964f-4f0c305f9537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:17.433697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1074/2150] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:17.441544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c199962-bc3e-4152-bea5-cfc8e096be39 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.441849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c199962-bc3e-4152-bea5-cfc8e096be39 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.442158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c199962-bc3e-4152-bea5-cfc8e096be39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.460667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7c199962-bc3e-4152-bea5-cfc8e096be39 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:17.461951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7c199962-bc3e-4152-bea5-cfc8e096be39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:17.470159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1077/2151] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:17.474925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ab2e8-e5eb-4ff4-964f-4f0c305f9537 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:17.488143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ab2e8-e5eb-4ff4-964f-4f0c305f9537 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:17.493629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a6c92-8852-4a95-9af0-ee3953d47630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:17.494305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a6c92-8852-4a95-9af0-ee3953d47630 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.494592 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a6c92-8852-4a95-9af0-ee3953d47630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.494873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a6c92-8852-4a95-9af0-ee3953d47630 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.495638 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a6c92-8852-4a95-9af0-ee3953d47630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:09:17.495949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a6c92-8852-4a95-9af0-ee3953d47630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:17.497616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a6c92-8852-4a95-9af0-ee3953d47630 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:17.505323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:17.511787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1075/2152] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:09:17.517298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c199962-bc3e-4152-bea5-cfc8e096be39 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:17.527264 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c199962-bc3e-4152-bea5-cfc8e096be39 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:17.532187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a948827-c02e-4c88-8418-43862786af72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:17.532896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a948827-c02e-4c88-8418-43862786af72 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.533176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a948827-c02e-4c88-8418-43862786af72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.533448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a948827-c02e-4c88-8418-43862786af72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.534149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a948827-c02e-4c88-8418-43862786af72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:17.536116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a948827-c02e-4c88-8418-43862786af72 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:17.567722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1078/2153] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:17 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:17.570222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a6c92-8852-4a95-9af0-ee3953d47630 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:17.581454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a6c92-8852-4a95-9af0-ee3953d47630 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:17.642174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483da4f-0796-4e0c-97cb-76b888e4b230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:17.642856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483da4f-0796-4e0c-97cb-76b888e4b230 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.643140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483da4f-0796-4e0c-97cb-76b888e4b230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.643443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483da4f-0796-4e0c-97cb-76b888e4b230 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.644174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483da4f-0796-4e0c-97cb-76b888e4b230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:17.646192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483da4f-0796-4e0c-97cb-76b888e4b230 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:17.668162 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1076/2154] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:17 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:17.672372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a948827-c02e-4c88-8418-43862786af72 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:17.683576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a948827-c02e-4c88-8418-43862786af72 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:17.688675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d29409-bf78-497f-ba13-cca4c594f0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:17.689362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d29409-bf78-497f-ba13-cca4c594f0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.689649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d29409-bf78-497f-ba13-cca4c594f0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.689930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d29409-bf78-497f-ba13-cca4c594f0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.690699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d29409-bf78-497f-ba13-cca4c594f0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:17.692492 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d29409-bf78-497f-ba13-cca4c594f0b2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:17.704379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1079/2155] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:09:17.708601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483da4f-0796-4e0c-97cb-76b888e4b230 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:17.719767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483da4f-0796-4e0c-97cb-76b888e4b230 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:17.725151 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97d77d-2c7e-4fca-acfc-f92616aa4aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:17.725767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97d77d-2c7e-4fca-acfc-f92616aa4aaa admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.726057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97d77d-2c7e-4fca-acfc-f92616aa4aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.726339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97d77d-2c7e-4fca-acfc-f92616aa4aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.727766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97d77d-2c7e-4fca-acfc-f92616aa4aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:17.729393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97d77d-2c7e-4fca-acfc-f92616aa4aaa admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:17.744745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1077/2156] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:17.749348 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d29409-bf78-497f-ba13-cca4c594f0b2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:17.765632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d29409-bf78-497f-ba13-cca4c594f0b2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:17.770615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987c7f8-4c29-42b2-b9b9-5f6182d5f569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:17.771714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987c7f8-4c29-42b2-b9b9-5f6182d5f569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.771895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987c7f8-4c29-42b2-b9b9-5f6182d5f569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.772220 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987c7f8-4c29-42b2-b9b9-5f6182d5f569 admin admin] PATH_INFO: `/v3/projects/55221024efb94951b9ed0c24567cf891/users/c522e39bfa734c9fa905578bee3273cd/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.785759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:17.790760 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987c7f8-4c29-42b2-b9b9-5f6182d5f569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55221024efb94951b9ed0c24567cf891, user_id=c522e39bfa734c9fa905578bee3273cd, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:17.794092 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987c7f8-4c29-42b2-b9b9-5f6182d5f569 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:17.809662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1080/2157] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:17 2022] PUT /identity/v3/projects/55221024efb94951b9ed0c24567cf891/users/c522e39bfa734c9fa905578bee3273cd/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:17.822120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f132fa-2be8-4255-8e46-7186db7650e8 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:17.822419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f132fa-2be8-4255-8e46-7186db7650e8 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:17.822822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f132fa-2be8-4255-8e46-7186db7650e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:17.858739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-30f132fa-2be8-4255-8e46-7186db7650e8 None None] MFA Rules not processed for user `c522e39bfa734c9fa905578bee3273cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:17.859986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-30f132fa-2be8-4255-8e46-7186db7650e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:17.939210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1078/2158] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:17.944168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987c7f8-4c29-42b2-b9b9-5f6182d5f569 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:18.035042 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987c7f8-4c29-42b2-b9b9-5f6182d5f569 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:18.040557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29744339-a638-4876-814b-0a292b84ce7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:18.041253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29744339-a638-4876-814b-0a292b84ce7e admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:18.041542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29744339-a638-4876-814b-0a292b84ce7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:18.041827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29744339-a638-4876-814b-0a292b84ce7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:18.042538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29744339-a638-4876-814b-0a292b84ce7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:18.044552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29744339-a638-4876-814b-0a292b84ce7e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:18.073222 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1081/2159] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:17 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:18.077773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f132fa-2be8-4255-8e46-7186db7650e8 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:18.088990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f132fa-2be8-4255-8e46-7186db7650e8 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:18.153332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e2816-e7ef-4b5c-9fe8-176ebd311676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:18.154022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2816-e7ef-4b5c-9fe8-176ebd311676 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:18.154319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2816-e7ef-4b5c-9fe8-176ebd311676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:18.154607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2816-e7ef-4b5c-9fe8-176ebd311676 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:18.155310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e2816-e7ef-4b5c-9fe8-176ebd311676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:18.157293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e2816-e7ef-4b5c-9fe8-176ebd311676 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:18.168029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:18.184247 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1079/2160] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:18 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:18.185748 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29744339-a638-4876-814b-0a292b84ce7e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:18.197624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29744339-a638-4876-814b-0a292b84ce7e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:18.203428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7fbf9c-0600-4850-86d6-aa6aa7caa29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:18.204121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fbf9c-0600-4850-86d6-aa6aa7caa29b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:18.204437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fbf9c-0600-4850-86d6-aa6aa7caa29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:18.204729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fbf9c-0600-4850-86d6-aa6aa7caa29b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:18.205637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7fbf9c-0600-4850-86d6-aa6aa7caa29b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:18.207267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7fbf9c-0600-4850-86d6-aa6aa7caa29b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:18.221389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1082/2161] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:18.226395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e2816-e7ef-4b5c-9fe8-176ebd311676 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:18.241114 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e2816-e7ef-4b5c-9fe8-176ebd311676 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:18.250426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840aa8e-badc-4d3c-ba5a-a8e84c23f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:18.250705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840aa8e-badc-4d3c-ba5a-a8e84c23f772 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:18.250802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840aa8e-badc-4d3c-ba5a-a8e84c23f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:18.251050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840aa8e-badc-4d3c-ba5a-a8e84c23f772 admin admin] PATH_INFO: `/v3/projects/38f8942b9fb84009b9fb7ef8d3692a3e/users/81d14413fe7d41819d1816b6677836c7/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:18.269648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840aa8e-badc-4d3c-ba5a-a8e84c23f772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f8942b9fb84009b9fb7ef8d3692a3e, user_id=81d14413fe7d41819d1816b6677836c7, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:18.272507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840aa8e-badc-4d3c-ba5a-a8e84c23f772 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:18.290481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1080/2162] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:18 2022] PUT /identity/v3/projects/38f8942b9fb84009b9fb7ef8d3692a3e/users/81d14413fe7d41819d1816b6677836c7/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:18.294531 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7fbf9c-0600-4850-86d6-aa6aa7caa29b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:18.306060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7fbf9c-0600-4850-86d6-aa6aa7caa29b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:18.382655 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dd6ba-e466-4a9e-b3a1-fbdc9bbc9dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:18.383362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dd6ba-e466-4a9e-b3a1-fbdc9bbc9dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:18.383684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dd6ba-e466-4a9e-b3a1-fbdc9bbc9dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:18.383969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dd6ba-e466-4a9e-b3a1-fbdc9bbc9dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:18.384696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dd6ba-e466-4a9e-b3a1-fbdc9bbc9dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:18.386308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dd6ba-e466-4a9e-b3a1-fbdc9bbc9dfb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:18.396486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1083/2163] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:18.401220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840aa8e-badc-4d3c-ba5a-a8e84c23f772 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:18.411813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840aa8e-badc-4d3c-ba5a-a8e84c23f772 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:18.417082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd2bf3-088a-46ff-9e2c-b5901acb8640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:18.417965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd2bf3-088a-46ff-9e2c-b5901acb8640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:18.418248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd2bf3-088a-46ff-9e2c-b5901acb8640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:18.418523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd2bf3-088a-46ff-9e2c-b5901acb8640 admin admin] PATH_INFO: `/v3/projects/38f8942b9fb84009b9fb7ef8d3692a3e/users/81d14413fe7d41819d1816b6677836c7/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:18.420530 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd2bf3-088a-46ff-9e2c-b5901acb8640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f8942b9fb84009b9fb7ef8d3692a3e, user_id=81d14413fe7d41819d1816b6677836c7, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:18.423164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd2bf3-088a-46ff-9e2c-b5901acb8640 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:18.433950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1081/2164] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:18 2022] PUT /identity/v3/projects/38f8942b9fb84009b9fb7ef8d3692a3e/users/81d14413fe7d41819d1816b6677836c7/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:18.448108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d50cc4-7d54-483e-b8ee-fa12fcd0ecb0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:18.448512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d50cc4-7d54-483e-b8ee-fa12fcd0ecb0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:18.448915 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d50cc4-7d54-483e-b8ee-fa12fcd0ecb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:18.476058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e3d50cc4-7d54-483e-b8ee-fa12fcd0ecb0 None None] MFA Rules not processed for user `81d14413fe7d41819d1816b6677836c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:18.477889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e3d50cc4-7d54-483e-b8ee-fa12fcd0ecb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:18.524753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:18.550847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1084/2165] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:18.563132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd2bf3-088a-46ff-9e2c-b5901acb8640 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:18.565174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8fbd2bf3-088a-46ff-9e2c-b5901acb8640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:18.623774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd2bf3-088a-46ff-9e2c-b5901acb8640 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:18.627979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66aebcf-3ce2-4648-9b43-9f35126264a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:18.628631 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66aebcf-3ce2-4648-9b43-9f35126264a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:18.628914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66aebcf-3ce2-4648-9b43-9f35126264a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:18.629193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66aebcf-3ce2-4648-9b43-9f35126264a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:18.688608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66aebcf-3ce2-4648-9b43-9f35126264a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:18.690058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66aebcf-3ce2-4648-9b43-9f35126264a9 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:18.699989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1082/2166] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:22.313624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d50cc4-7d54-483e-b8ee-fa12fcd0ecb0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:22.386303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d50cc4-7d54-483e-b8ee-fa12fcd0ecb0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:22.391215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:22.392151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:22.392439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:22.392721 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] PATH_INFO: `/v3/users/c522e39bfa734c9fa905578bee3273cd` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:22.393929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c522e39bfa734c9fa905578bee3273cd)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:22.396144 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:22.448465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c522e39bfa734c9fa905578bee3273cd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:22.458876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c522e39bfa734c9fa905578bee3273cd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:22.460363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c522e39bfa734c9fa905578bee3273cd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:22.470093 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c522e39bfa734c9fa905578bee3273cd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:22.470986 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c522e39bfa734c9fa905578bee3273cd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:22.499718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1085/2167] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:22 2022] DELETE /identity/v3/users/c522e39bfa734c9fa905578bee3273cd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:22.506902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66233c2b-ae7b-4a05-96de-28365c5ab7bd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:22.507866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66233c2b-ae7b-4a05-96de-28365c5ab7bd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:22.509078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66233c2b-ae7b-4a05-96de-28365c5ab7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:22.541852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-66233c2b-ae7b-4a05-96de-28365c5ab7bd None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:22.543133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-66233c2b-ae7b-4a05-96de-28365c5ab7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:22.646660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1083/2168] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:22.653448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:22.654899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:22.716398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cefa20-d65e-40ad-8fb0-f8388f42a425 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:22.720283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a0685-7bab-4069-9b22-f803e8e20710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:22.720971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a0685-7bab-4069-9b22-f803e8e20710 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:22.721363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a0685-7bab-4069-9b22-f803e8e20710 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:22.721696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a0685-7bab-4069-9b22-f803e8e20710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:22.732445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a0685-7bab-4069-9b22-f803e8e20710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:22.733816 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a0685-7bab-4069-9b22-f803e8e20710 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:22.744901 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1086/2169] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:23.117821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66233c2b-ae7b-4a05-96de-28365c5ab7bd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:23.118767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-66233c2b-ae7b-4a05-96de-28365c5ab7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:23.143445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66233c2b-ae7b-4a05-96de-28365c5ab7bd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:23.151509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:23.152429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:23.152599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:23.152874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] PATH_INFO: `/v3/projects/55221024efb94951b9ed0c24567cf891` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:23.154281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55221024efb94951b9ed0c24567cf891)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:23.156424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:23.168995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:23.238992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55221024efb94951b9ed0c24567cf891', 'request_id': 'req-274b70cd-ae31-4e37-84f5-ab061aa4af45'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:23.247562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55221024efb94951b9ed0c24567cf891', 'request_id': 'req-274b70cd-ae31-4e37-84f5-ab061aa4af45'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:23.255271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55221024efb94951b9ed0c24567cf891', 'request_id': 'req-274b70cd-ae31-4e37-84f5-ab061aa4af45'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:23.283463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] The token cache is being invalidate because project 55221024efb94951b9ed0c24567cf891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:23.283767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:23.284490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:23.287717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1084/2170] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:23 2022] DELETE /identity/v3/projects/55221024efb94951b9ed0c24567cf891 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:23.292413 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a0685-7bab-4069-9b22-f803e8e20710 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:23.310995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a0685-7bab-4069-9b22-f803e8e20710 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:23.315635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:23.386868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b7744-5386-455f-b445-df48ba6ea3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:23.386868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b7744-5386-455f-b445-df48ba6ea3b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:23.387594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b7744-5386-455f-b445-df48ba6ea3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:23.388041 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b7744-5386-455f-b445-df48ba6ea3b4 admin admin] PATH_INFO: `/v3/users/81d14413fe7d41819d1816b6677836c7` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:23.389469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b7744-5386-455f-b445-df48ba6ea3b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d14413fe7d41819d1816b6677836c7)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:23.391483 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b7744-5386-455f-b445-df48ba6ea3b4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:23.449828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-077b7744-5386-455f-b445-df48ba6ea3b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d14413fe7d41819d1816b6677836c7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:23.458705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-077b7744-5386-455f-b445-df48ba6ea3b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d14413fe7d41819d1816b6677836c7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:23.459472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-077b7744-5386-455f-b445-df48ba6ea3b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d14413fe7d41819d1816b6677836c7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:23.470736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-077b7744-5386-455f-b445-df48ba6ea3b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d14413fe7d41819d1816b6677836c7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:23.471831 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-077b7744-5386-455f-b445-df48ba6ea3b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d14413fe7d41819d1816b6677836c7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:23.495227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1087/2171] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:23 2022] DELETE /identity/v3/users/81d14413fe7d41819d1816b6677836c7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:23.963673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:23.965166 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:23.991624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b70cd-ae31-4e37-84f5-ab061aa4af45 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:24.070327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98fe2d-62db-4d38-917c-af34a8a84c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:24.071210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98fe2d-62db-4d38-917c-af34a8a84c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:24.071525 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98fe2d-62db-4d38-917c-af34a8a84c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:24.071806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98fe2d-62db-4d38-917c-af34a8a84c2e admin admin] PATH_INFO: `/v3/projects/38f8942b9fb84009b9fb7ef8d3692a3e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:24.073083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98fe2d-62db-4d38-917c-af34a8a84c2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=38f8942b9fb84009b9fb7ef8d3692a3e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:24.075027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98fe2d-62db-4d38-917c-af34a8a84c2e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:24.139635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9a98fe2d-62db-4d38-917c-af34a8a84c2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38f8942b9fb84009b9fb7ef8d3692a3e', 'request_id': 'req-9a98fe2d-62db-4d38-917c-af34a8a84c2e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:24.146074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9a98fe2d-62db-4d38-917c-af34a8a84c2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38f8942b9fb84009b9fb7ef8d3692a3e', 'request_id': 'req-9a98fe2d-62db-4d38-917c-af34a8a84c2e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:24.152281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9a98fe2d-62db-4d38-917c-af34a8a84c2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38f8942b9fb84009b9fb7ef8d3692a3e', 'request_id': 'req-9a98fe2d-62db-4d38-917c-af34a8a84c2e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:24.168749 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9a98fe2d-62db-4d38-917c-af34a8a84c2e admin admin] The token cache is being invalidate because project 38f8942b9fb84009b9fb7ef8d3692a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:24.169054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9a98fe2d-62db-4d38-917c-af34a8a84c2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:24.170107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9a98fe2d-62db-4d38-917c-af34a8a84c2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:24.172045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1085/2172] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:23 2022] DELETE /identity/v3/projects/38f8942b9fb84009b9fb7ef8d3692a3e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:24.177297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b4456-930c-4ac9-86f8-9229a9042e59 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:24.177693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b4456-930c-4ac9-86f8-9229a9042e59 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:24.178207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b4456-930c-4ac9-86f8-9229a9042e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:24.219598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-016b4456-930c-4ac9-86f8-9229a9042e59 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:24.219598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-016b4456-930c-4ac9-86f8-9229a9042e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:24.298381 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1088/2173] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:24.303177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e2df2-8d85-430c-af49-e606951c177b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:24.303616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e2df2-8d85-430c-af49-e606951c177b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:24.304036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e2df2-8d85-430c-af49-e606951c177b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:24.325258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2f0e2df2-8d85-430c-af49-e606951c177b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:24.326553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2f0e2df2-8d85-430c-af49-e606951c177b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:24.338759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1086/2174] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:24.340031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b4456-930c-4ac9-86f8-9229a9042e59 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:24.353095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b4456-930c-4ac9-86f8-9229a9042e59 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:24.358814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02996-7428-4074-90c8-fad52465636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:24.359530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02996-7428-4074-90c8-fad52465636f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:24.359818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02996-7428-4074-90c8-fad52465636f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:24.360097 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da02996-7428-4074-90c8-fad52465636f admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:24.360851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da02996-7428-4074-90c8-fad52465636f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:09:24.361163 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da02996-7428-4074-90c8-fad52465636f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:24.362847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da02996-7428-4074-90c8-fad52465636f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:24.378309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1089/2175] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:09:24.382856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e2df2-8d85-430c-af49-e606951c177b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:24.396947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e2df2-8d85-430c-af49-e606951c177b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:24.402834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d059c-659c-430a-960d-110fb91b1de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:24.403582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d059c-659c-430a-960d-110fb91b1de9 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:24.403880 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d059c-659c-430a-960d-110fb91b1de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:24.404139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d059c-659c-430a-960d-110fb91b1de9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:24.404858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d059c-659c-430a-960d-110fb91b1de9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:24.406826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d059c-659c-430a-960d-110fb91b1de9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:24.430728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:24.446422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1087/2176] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:24 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:09:24.447753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02996-7428-4074-90c8-fad52465636f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:24.458469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da02996-7428-4074-90c8-fad52465636f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:24.473925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:24.535115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5098f90-3e97-410d-9057-0845a4652bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:24.535764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5098f90-3e97-410d-9057-0845a4652bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:24.536052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5098f90-3e97-410d-9057-0845a4652bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:24.536332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5098f90-3e97-410d-9057-0845a4652bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:24.537058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5098f90-3e97-410d-9057-0845a4652bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:24.539809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5098f90-3e97-410d-9057-0845a4652bd0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:24.562444 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1090/2177] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:24 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:09:24.567138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d059c-659c-430a-960d-110fb91b1de9 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:24.578719 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d059c-659c-430a-960d-110fb91b1de9 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:24.583888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea4e9a-63b8-4fc3-9fd1-30b0447a22c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:24.584584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea4e9a-63b8-4fc3-9fd1-30b0447a22c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:24.584875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea4e9a-63b8-4fc3-9fd1-30b0447a22c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:24.585161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea4e9a-63b8-4fc3-9fd1-30b0447a22c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:24.586489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea4e9a-63b8-4fc3-9fd1-30b0447a22c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:24.588191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea4e9a-63b8-4fc3-9fd1-30b0447a22c0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:24.600899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1088/2178] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:09:24.605562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5098f90-3e97-410d-9057-0845a4652bd0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:24.617490 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5098f90-3e97-410d-9057-0845a4652bd0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:24.624578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b95033-7eb0-4aae-8aa1-2e642b1fa77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:24.625350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b95033-7eb0-4aae-8aa1-2e642b1fa77b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:24.625636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b95033-7eb0-4aae-8aa1-2e642b1fa77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:24.625969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b95033-7eb0-4aae-8aa1-2e642b1fa77b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:24.626753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b95033-7eb0-4aae-8aa1-2e642b1fa77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:24.628698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b95033-7eb0-4aae-8aa1-2e642b1fa77b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:24.640337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1091/2179] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:09:24.644866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea4e9a-63b8-4fc3-9fd1-30b0447a22c0 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:24.657417 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea4e9a-63b8-4fc3-9fd1-30b0447a22c0 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:24.662283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322319b8-778f-40d9-936e-c90c5df5bcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:24.663180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322319b8-778f-40d9-936e-c90c5df5bcaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:24.663497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322319b8-778f-40d9-936e-c90c5df5bcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:24.663776 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322319b8-778f-40d9-936e-c90c5df5bcaa admin admin] PATH_INFO: `/v3/projects/6e18d83c0c834079bd6520c2eae75dd6/users/047e86053d2c4f41b1f1680a7b259995/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:24.684382 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322319b8-778f-40d9-936e-c90c5df5bcaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e18d83c0c834079bd6520c2eae75dd6, user_id=047e86053d2c4f41b1f1680a7b259995, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:24.693993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322319b8-778f-40d9-936e-c90c5df5bcaa admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:24.709926 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1089/2180] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:24 2022] PUT /identity/v3/projects/6e18d83c0c834079bd6520c2eae75dd6/users/047e86053d2c4f41b1f1680a7b259995/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:24.722853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09789e89-c7a4-4208-bbab-d049c8284849 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:24.723159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09789e89-c7a4-4208-bbab-d049c8284849 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:24.723453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09789e89-c7a4-4208-bbab-d049c8284849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:24.753247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-09789e89-c7a4-4208-bbab-d049c8284849 None None] MFA Rules not processed for user `047e86053d2c4f41b1f1680a7b259995`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:24.754500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-09789e89-c7a4-4208-bbab-d049c8284849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:24.835967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1092/2181] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:24.841051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322319b8-778f-40d9-936e-c90c5df5bcaa admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:24.932104 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322319b8-778f-40d9-936e-c90c5df5bcaa admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:24.938132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001c33c-29a0-4999-b602-deee770d36df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:24.938828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001c33c-29a0-4999-b602-deee770d36df admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:24.939115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001c33c-29a0-4999-b602-deee770d36df admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:24.939480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001c33c-29a0-4999-b602-deee770d36df admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:24.940261 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001c33c-29a0-4999-b602-deee770d36df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:24.942198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001c33c-29a0-4999-b602-deee770d36df admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:24.983618 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1090/2182] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:24 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:09:24.988662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09789e89-c7a4-4208-bbab-d049c8284849 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:25.005149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09789e89-c7a4-4208-bbab-d049c8284849 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:25.087472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6aef23-68ab-4803-bb01-0f86fba9aad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:25.088669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aef23-68ab-4803-bb01-0f86fba9aad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.088942 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aef23-68ab-4803-bb01-0f86fba9aad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.089226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aef23-68ab-4803-bb01-0f86fba9aad4 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.090151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6aef23-68ab-4803-bb01-0f86fba9aad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:25.092073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6aef23-68ab-4803-bb01-0f86fba9aad4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:25.116586 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1093/2183] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:24 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:25.121334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001c33c-29a0-4999-b602-deee770d36df admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:25.135623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001c33c-29a0-4999-b602-deee770d36df admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:25.139738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad096cb6-2381-437a-8295-0769483759cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:25.140673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad096cb6-2381-437a-8295-0769483759cd admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.140963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad096cb6-2381-437a-8295-0769483759cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.141258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad096cb6-2381-437a-8295-0769483759cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.142000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad096cb6-2381-437a-8295-0769483759cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:25.143662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad096cb6-2381-437a-8295-0769483759cd admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:25.160630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1091/2184] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:25.167070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6aef23-68ab-4803-bb01-0f86fba9aad4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:25.179874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6aef23-68ab-4803-bb01-0f86fba9aad4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:25.184998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2af427-4775-4495-873c-8d721144991f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:25.186225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2af427-4775-4495-873c-8d721144991f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.186225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2af427-4775-4495-873c-8d721144991f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.186225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2af427-4775-4495-873c-8d721144991f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.186962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2af427-4775-4495-873c-8d721144991f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:25.188673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2af427-4775-4495-873c-8d721144991f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:25.202988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:25.205896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1094/2185] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:09:25.207618 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad096cb6-2381-437a-8295-0769483759cd admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:25.218911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad096cb6-2381-437a-8295-0769483759cd admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:25.223644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcef005-4ef2-4725-a6e4-a585e326685f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:25.224531 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcef005-4ef2-4725-a6e4-a585e326685f admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.224816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcef005-4ef2-4725-a6e4-a585e326685f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.225093 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcef005-4ef2-4725-a6e4-a585e326685f admin admin] PATH_INFO: `/v3/projects/ca21978c2f044cf9b2f21e0fc058a52a/users/90d53d0ec2204f73b96a2309138786ea/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.241815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcef005-4ef2-4725-a6e4-a585e326685f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca21978c2f044cf9b2f21e0fc058a52a, user_id=90d53d0ec2204f73b96a2309138786ea, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:25.245305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcef005-4ef2-4725-a6e4-a585e326685f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:25.260361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1092/2186] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:25 2022] PUT /identity/v3/projects/ca21978c2f044cf9b2f21e0fc058a52a/users/90d53d0ec2204f73b96a2309138786ea/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:25.271801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d06999-3f64-43d0-96fc-78fceccc993c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.272094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d06999-3f64-43d0-96fc-78fceccc993c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.272522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d06999-3f64-43d0-96fc-78fceccc993c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.303371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a5d06999-3f64-43d0-96fc-78fceccc993c None None] MFA Rules not processed for user `90d53d0ec2204f73b96a2309138786ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:25.304932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a5d06999-3f64-43d0-96fc-78fceccc993c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:25.373291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1095/2187] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:25.378194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcef005-4ef2-4725-a6e4-a585e326685f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:25.463559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcef005-4ef2-4725-a6e4-a585e326685f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:25.468786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf331c-440e-4e4b-973a-3df68580c646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:25.469478 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf331c-440e-4e4b-973a-3df68580c646 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.469770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf331c-440e-4e4b-973a-3df68580c646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.470057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf331c-440e-4e4b-973a-3df68580c646 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.470773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf331c-440e-4e4b-973a-3df68580c646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:25.473127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf331c-440e-4e4b-973a-3df68580c646 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:25.510544 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:25.520451 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1093/2188] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:25 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:25.521692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d06999-3f64-43d0-96fc-78fceccc993c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:25.535778 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d06999-3f64-43d0-96fc-78fceccc993c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:25.627582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80c86e-47b7-4fac-b8c2-fda46b687df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:25.628240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80c86e-47b7-4fac-b8c2-fda46b687df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.628528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80c86e-47b7-4fac-b8c2-fda46b687df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.628815 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80c86e-47b7-4fac-b8c2-fda46b687df5 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.629578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80c86e-47b7-4fac-b8c2-fda46b687df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:25.631679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80c86e-47b7-4fac-b8c2-fda46b687df5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:25.660791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1096/2189] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:25 2022] POST /identity/v3/users => generated 295 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:25.664742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf331c-440e-4e4b-973a-3df68580c646 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:25.675815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf331c-440e-4e4b-973a-3df68580c646 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:25.681644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d1761-f552-4fb3-803a-bdf4822a7f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:25.682588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d1761-f552-4fb3-803a-bdf4822a7f84 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.682976 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d1761-f552-4fb3-803a-bdf4822a7f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.683359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d1761-f552-4fb3-803a-bdf4822a7f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.684238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d1761-f552-4fb3-803a-bdf4822a7f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:25.686145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d1761-f552-4fb3-803a-bdf4822a7f84 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:25.705297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1094/2190] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:25.706458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80c86e-47b7-4fac-b8c2-fda46b687df5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:25.719970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80c86e-47b7-4fac-b8c2-fda46b687df5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:25.725110 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea101bd-2941-4e3a-89e5-7f86aa43095e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:25.726030 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea101bd-2941-4e3a-89e5-7f86aa43095e admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.726377 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea101bd-2941-4e3a-89e5-7f86aa43095e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.726668 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea101bd-2941-4e3a-89e5-7f86aa43095e admin admin] PATH_INFO: `/v3/projects/768738017c28424aab490f8d79d9dace/users/662920c0c920494796cc98de7e5fe334/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.744111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea101bd-2941-4e3a-89e5-7f86aa43095e admin admin] RBAC: Authorizing `identity:create_grant(project_id=768738017c28424aab490f8d79d9dace, user_id=662920c0c920494796cc98de7e5fe334, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:25.746884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea101bd-2941-4e3a-89e5-7f86aa43095e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:25.767709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1097/2191] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:25 2022] PUT /identity/v3/projects/768738017c28424aab490f8d79d9dace/users/662920c0c920494796cc98de7e5fe334/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:25.772115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d1761-f552-4fb3-803a-bdf4822a7f84 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:25.797516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d1761-f552-4fb3-803a-bdf4822a7f84 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:25.896453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f001e2d-9fea-49cd-bde2-28256bfdc6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:25.897320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f001e2d-9fea-49cd-bde2-28256bfdc6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.897698 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f001e2d-9fea-49cd-bde2-28256bfdc6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.898073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f001e2d-9fea-49cd-bde2-28256bfdc6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.899570 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f001e2d-9fea-49cd-bde2-28256bfdc6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:25.901440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f001e2d-9fea-49cd-bde2-28256bfdc6f3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:25.912409 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1095/2192] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:25.918022 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea101bd-2941-4e3a-89e5-7f86aa43095e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:25.937142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea101bd-2941-4e3a-89e5-7f86aa43095e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:25.945799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8a077-a1ed-45aa-a4dc-d8340890b11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:25.947215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8a077-a1ed-45aa-a4dc-d8340890b11b admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:25.947716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8a077-a1ed-45aa-a4dc-d8340890b11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:25.948215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8a077-a1ed-45aa-a4dc-d8340890b11b admin admin] PATH_INFO: `/v3/projects/768738017c28424aab490f8d79d9dace/users/662920c0c920494796cc98de7e5fe334/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:25.953535 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8a077-a1ed-45aa-a4dc-d8340890b11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=768738017c28424aab490f8d79d9dace, user_id=662920c0c920494796cc98de7e5fe334, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:25.959970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8a077-a1ed-45aa-a4dc-d8340890b11b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:25.983938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1098/2193] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:25 2022] PUT /identity/v3/projects/768738017c28424aab490f8d79d9dace/users/662920c0c920494796cc98de7e5fe334/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:26.005438 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b89fe-4cff-44b3-a683-aabc2dd0e387 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:26.005993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b89fe-4cff-44b3-a683-aabc2dd0e387 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:26.006495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b89fe-4cff-44b3-a683-aabc2dd0e387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:26.060902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-458b89fe-4cff-44b3-a683-aabc2dd0e387 None None] MFA Rules not processed for user `662920c0c920494796cc98de7e5fe334`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:26.063031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-458b89fe-4cff-44b3-a683-aabc2dd0e387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:26.157278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1096/2194] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:26.169415 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8a077-a1ed-45aa-a4dc-d8340890b11b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:26.173849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c4c8a077-a1ed-45aa-a4dc-d8340890b11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:26.256465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:26.278955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8a077-a1ed-45aa-a4dc-d8340890b11b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:26.286669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d2859-75c7-4b4d-a71a-5807aecea0ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:26.287613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d2859-75c7-4b4d-a71a-5807aecea0ff service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:26.288038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d2859-75c7-4b4d-a71a-5807aecea0ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:26.288434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d2859-75c7-4b4d-a71a-5807aecea0ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:26.392692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d2859-75c7-4b4d-a71a-5807aecea0ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:26.394440 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d2859-75c7-4b4d-a71a-5807aecea0ff service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:26.411487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1099/2195] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:28.039210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b89fe-4cff-44b3-a683-aabc2dd0e387 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:28.050956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b89fe-4cff-44b3-a683-aabc2dd0e387 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:28.055586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb90ff-f6c0-418c-b220-1f8c487ad82c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:28.056287 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb90ff-f6c0-418c-b220-1f8c487ad82c service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:28.056576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb90ff-f6c0-418c-b220-1f8c487ad82c service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:28.056961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb90ff-f6c0-418c-b220-1f8c487ad82c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:28.128061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb90ff-f6c0-418c-b220-1f8c487ad82c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:28.129521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb90ff-f6c0-418c-b220-1f8c487ad82c service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:28.140383 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1097/2196] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:28.933199 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc8b0-d116-4019-80e1-74e8fab6d87a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:28.934226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc8b0-d116-4019-80e1-74e8fab6d87a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:28.935035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc8b0-d116-4019-80e1-74e8fab6d87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:28.966662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0adbc8b0-d116-4019-80e1-74e8fab6d87a None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:28.969412 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0adbc8b0-d116-4019-80e1-74e8fab6d87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:28.982006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb90ff-f6c0-418c-b220-1f8c487ad82c service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:28.994010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb90ff-f6c0-418c-b220-1f8c487ad82c service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:28.999121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646d7de-9be9-4cba-986a-6254d3ffa4c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:29.000164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646d7de-9be9-4cba-986a-6254d3ffa4c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:29.000460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646d7de-9be9-4cba-986a-6254d3ffa4c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:29.000739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646d7de-9be9-4cba-986a-6254d3ffa4c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:29.025125 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:29.033376 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646d7de-9be9-4cba-986a-6254d3ffa4c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:29.034943 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646d7de-9be9-4cba-986a-6254d3ffa4c3 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:29.068501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1098/2197] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:29.084440 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1100/2198] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:09:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:29.138069 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646d7de-9be9-4cba-986a-6254d3ffa4c3 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:29.140165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7646d7de-9be9-4cba-986a-6254d3ffa4c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:29.246293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646d7de-9be9-4cba-986a-6254d3ffa4c3 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:29.252857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1660f4-2f96-45c4-bfa9-99f0c8c33bc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:29.253599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1660f4-2f96-45c4-bfa9-99f0c8c33bc6 service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:29.253976 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1660f4-2f96-45c4-bfa9-99f0c8c33bc6 service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:29.254826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1660f4-2f96-45c4-bfa9-99f0c8c33bc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:29.265808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1660f4-2f96-45c4-bfa9-99f0c8c33bc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:29.268713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1660f4-2f96-45c4-bfa9-99f0c8c33bc6 service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:29.278961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1099/2199] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:09:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 11:09:30.079522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81646b-a7ad-4f0a-a75c-3e9e14c7f5c5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:30.080055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81646b-a7ad-4f0a-a75c-3e9e14c7f5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:30.080688 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81646b-a7ad-4f0a-a75c-3e9e14c7f5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:30.105858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-af81646b-a7ad-4f0a-a75c-3e9e14c7f5c5 None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:30.107462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-af81646b-a7ad-4f0a-a75c-3e9e14c7f5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:30.130262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1101/2200] 10.222.0.75 () {58 vars in 967 bytes} [Thu Jul 14 11:09:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:30.130651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1660f4-2f96-45c4-bfa9-99f0c8c33bc6 service swift] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:30.146847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1660f4-2f96-45c4-bfa9-99f0c8c33bc6 service swift] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:30.158359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dc74a-acf6-4fb2-afaf-0c1030906bd6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:30.159228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dc74a-acf6-4fb2-afaf-0c1030906bd6 service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:30.159571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dc74a-acf6-4fb2-afaf-0c1030906bd6 service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:30.159852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dc74a-acf6-4fb2-afaf-0c1030906bd6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:30.173259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dc74a-acf6-4fb2-afaf-0c1030906bd6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:30.175198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dc74a-acf6-4fb2-afaf-0c1030906bd6 service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:30.201232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1100/2201] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:09:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:09:30.643433 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65b7b9-be2e-43d8-aabb-3dddce7f90e1 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:30.643734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65b7b9-be2e-43d8-aabb-3dddce7f90e1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:30.644038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65b7b9-be2e-43d8-aabb-3dddce7f90e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:30.676224 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-be65b7b9-be2e-43d8-aabb-3dddce7f90e1 None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:30.677534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-be65b7b9-be2e-43d8-aabb-3dddce7f90e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:30.691626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1102/2202] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:09:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:30.714347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dc74a-acf6-4fb2-afaf-0c1030906bd6 service swift] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:30.742833 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dc74a-acf6-4fb2-afaf-0c1030906bd6 service swift] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:30.754019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efb1f9-322b-4180-82a4-57d590abfe87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:30.755218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efb1f9-322b-4180-82a4-57d590abfe87 service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:30.766564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efb1f9-322b-4180-82a4-57d590abfe87 service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:30.766564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efb1f9-322b-4180-82a4-57d590abfe87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:30.777442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efb1f9-322b-4180-82a4-57d590abfe87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:30.779373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efb1f9-322b-4180-82a4-57d590abfe87 service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:30.799243 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1101/2203] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:09:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:09:31.072703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3031552-6bc6-470c-ad80-139d7f18aa66 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:31.072994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3031552-6bc6-470c-ad80-139d7f18aa66 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:31.073750 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3031552-6bc6-470c-ad80-139d7f18aa66 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:31.075187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1103/2204] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 11:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:09:31.088398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a153f8-f3f4-4b35-8504-d1b4c902a87a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:31.088398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a153f8-f3f4-4b35-8504-d1b4c902a87a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:31.088398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a153f8-f3f4-4b35-8504-d1b4c902a87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:31.112976 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-19a153f8-f3f4-4b35-8504-d1b4c902a87a None None] MFA Rules not processed for user `0fc1f2abce4845219741f4f48bf32d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:31.115880 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-19a153f8-f3f4-4b35-8504-d1b4c902a87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:31.161010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3031552-6bc6-470c-ad80-139d7f18aa66 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:31.164105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c3031552-6bc6-470c-ad80-139d7f18aa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:31.236788 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1102/2205] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 11:09:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:31.243875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bde19-93e4-4e4b-bdf9-4b4d0f617db4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:31.244176 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bde19-93e4-4e4b-bdf9-4b4d0f617db4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:31.244464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bde19-93e4-4e4b-bdf9-4b4d0f617db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:31.263701 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-930bde19-93e4-4e4b-bdf9-4b4d0f617db4 None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:31.264950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-930bde19-93e4-4e4b-bdf9-4b4d0f617db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:31.273210 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1103/2206] 10.222.0.75 () {58 vars in 967 bytes} [Thu Jul 14 11:09:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:31.274041 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bde19-93e4-4e4b-bdf9-4b4d0f617db4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:31.275374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-930bde19-93e4-4e4b-bdf9-4b4d0f617db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:31.279252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3031552-6bc6-470c-ad80-139d7f18aa66 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:31.287746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486f822-29a7-4c6c-a20b-9fbbe75a8167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:31.289321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486f822-29a7-4c6c-a20b-9fbbe75a8167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:31.289833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486f822-29a7-4c6c-a20b-9fbbe75a8167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:31.290345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486f822-29a7-4c6c-a20b-9fbbe75a8167 admin admin] PATH_INFO: `/v3/users/f330cdfd7d2b47a985d8dc76c19da9a4` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:31.292255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bde19-93e4-4e4b-bdf9-4b4d0f617db4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:31.293005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486f822-29a7-4c6c-a20b-9fbbe75a8167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f330cdfd7d2b47a985d8dc76c19da9a4)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:31.297069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486f822-29a7-4c6c-a20b-9fbbe75a8167 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:31.298059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db8bc0-8742-41d9-9b0c-fccc5e732c0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:31.298910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db8bc0-8742-41d9-9b0c-fccc5e732c0d service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:31.299318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db8bc0-8742-41d9-9b0c-fccc5e732c0d service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:31.299732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db8bc0-8742-41d9-9b0c-fccc5e732c0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:31.312687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db8bc0-8742-41d9-9b0c-fccc5e732c0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:31.316247 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db8bc0-8742-41d9-9b0c-fccc5e732c0d service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:31.338035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1104/2207] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:31.340284 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db8bc0-8742-41d9-9b0c-fccc5e732c0d service placement] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:31.346177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:31.356136 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db8bc0-8742-41d9-9b0c-fccc5e732c0d service placement] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:31.362575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518288b-a9ea-4c2a-9278-3695ad68c0ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:31.363327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518288b-a9ea-4c2a-9278-3695ad68c0ae service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:31.363680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518288b-a9ea-4c2a-9278-3695ad68c0ae service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:31.363926 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518288b-a9ea-4c2a-9278-3695ad68c0ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:31.367697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5486f822-29a7-4c6c-a20b-9fbbe75a8167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f330cdfd7d2b47a985d8dc76c19da9a4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:31.375988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518288b-a9ea-4c2a-9278-3695ad68c0ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:31.377265 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518288b-a9ea-4c2a-9278-3695ad68c0ae service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:31.378046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5486f822-29a7-4c6c-a20b-9fbbe75a8167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f330cdfd7d2b47a985d8dc76c19da9a4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:31.379066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5486f822-29a7-4c6c-a20b-9fbbe75a8167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f330cdfd7d2b47a985d8dc76c19da9a4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:31.387575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1105/2208] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:09:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:09:31.393610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5486f822-29a7-4c6c-a20b-9fbbe75a8167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f330cdfd7d2b47a985d8dc76c19da9a4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:31.394455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5486f822-29a7-4c6c-a20b-9fbbe75a8167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f330cdfd7d2b47a985d8dc76c19da9a4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:31.418343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1104/2209] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:31 2022] DELETE /identity/v3/users/f330cdfd7d2b47a985d8dc76c19da9a4 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:31.639789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518288b-a9ea-4c2a-9278-3695ad68c0ae service swift] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:31.657294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518288b-a9ea-4c2a-9278-3695ad68c0ae service swift] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:31.700004 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:31.759895 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:31.760902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:31.761057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:31.761510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0 admin admin] PATH_INFO: `/v3/projects/a90a870ee4154287be1282b3f46bdd05` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:31.762773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a90a870ee4154287be1282b3f46bdd05)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:31.764753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:31.850912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a90a870ee4154287be1282b3f46bdd05', 'request_id': 'req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:31.864183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a90a870ee4154287be1282b3f46bdd05', 'request_id': 'req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:31.878205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a90a870ee4154287be1282b3f46bdd05', 'request_id': 'req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:31.900677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0 admin admin] The token cache is being invalidate because project a90a870ee4154287be1282b3f46bdd05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:31.900983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:31.901823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fb6bc259-fcc3-4e82-9652-c0f70d3b78d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:31.910331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1106/2210] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:31 2022] DELETE /identity/v3/projects/a90a870ee4154287be1282b3f46bdd05 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:31.915575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf95da-4ab1-41a7-b77e-68fa16f8d5ea None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:31.915873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf95da-4ab1-41a7-b77e-68fa16f8d5ea None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:31.916157 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf95da-4ab1-41a7-b77e-68fa16f8d5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:31.938536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c8cf95da-4ab1-41a7-b77e-68fa16f8d5ea None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:31.939600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c8cf95da-4ab1-41a7-b77e-68fa16f8d5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:32.039354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1105/2211] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:32.047316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5049f-cc29-4cd1-977b-2cb123b443b0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:32.047742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5049f-cc29-4cd1-977b-2cb123b443b0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:32.048058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5049f-cc29-4cd1-977b-2cb123b443b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:32.072319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e8a5049f-cc29-4cd1-977b-2cb123b443b0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:32.073705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e8a5049f-cc29-4cd1-977b-2cb123b443b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:32.091608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1107/2212] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:32.092786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf95da-4ab1-41a7-b77e-68fa16f8d5ea None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:32.112288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf95da-4ab1-41a7-b77e-68fa16f8d5ea None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:32.118148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94435277-516d-4952-86c9-1d6607be86ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:32.118917 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94435277-516d-4952-86c9-1d6607be86ef admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:32.119208 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94435277-516d-4952-86c9-1d6607be86ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:32.119517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94435277-516d-4952-86c9-1d6607be86ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:32.120261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94435277-516d-4952-86c9-1d6607be86ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:09:32.120571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94435277-516d-4952-86c9-1d6607be86ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:32.122365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94435277-516d-4952-86c9-1d6607be86ef admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:32.139550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1106/2213] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:09:32.140762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5049f-cc29-4cd1-977b-2cb123b443b0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:32.152843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5049f-cc29-4cd1-977b-2cb123b443b0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:32.165024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcf577-3a43-40c1-b778-51f8adf84bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:32.166051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcf577-3a43-40c1-b778-51f8adf84bda admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:32.166436 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcf577-3a43-40c1-b778-51f8adf84bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:32.166789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcf577-3a43-40c1-b778-51f8adf84bda admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:32.167642 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcf577-3a43-40c1-b778-51f8adf84bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:32.172846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcf577-3a43-40c1-b778-51f8adf84bda admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:32.217663 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1108/2214] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:32 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:09:32.221214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94435277-516d-4952-86c9-1d6607be86ef admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:32.236598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94435277-516d-4952-86c9-1d6607be86ef admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:32.319476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61975322-0e89-4742-82ef-d8718343184f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:32.320724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61975322-0e89-4742-82ef-d8718343184f admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:32.321017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61975322-0e89-4742-82ef-d8718343184f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:32.321301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61975322-0e89-4742-82ef-d8718343184f admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:32.322021 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61975322-0e89-4742-82ef-d8718343184f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:32.324745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61975322-0e89-4742-82ef-d8718343184f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:32.371446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1107/2215] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:32 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:09:32.374640 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcf577-3a43-40c1-b778-51f8adf84bda admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:32.394323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcf577-3a43-40c1-b778-51f8adf84bda admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:32.403652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a1f66-3c11-4a6d-bee2-8a29a1056e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:32.403852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a1f66-3c11-4a6d-bee2-8a29a1056e76 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:32.404140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a1f66-3c11-4a6d-bee2-8a29a1056e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:32.404843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a1f66-3c11-4a6d-bee2-8a29a1056e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:32.405168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a1f66-3c11-4a6d-bee2-8a29a1056e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:32.406798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a1f66-3c11-4a6d-bee2-8a29a1056e76 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:32.423051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1109/2216] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:32.426995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61975322-0e89-4742-82ef-d8718343184f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:32.439817 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61975322-0e89-4742-82ef-d8718343184f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:32.445631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f2fa5-b4dd-40d7-a8ca-2af5554b2f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:32.446475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f2fa5-b4dd-40d7-a8ca-2af5554b2f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:32.446826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f2fa5-b4dd-40d7-a8ca-2af5554b2f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:32.447171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f2fa5-b4dd-40d7-a8ca-2af5554b2f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:32.448030 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f2fa5-b4dd-40d7-a8ca-2af5554b2f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:32.451567 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f2fa5-b4dd-40d7-a8ca-2af5554b2f14 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:32.467086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1108/2217] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:32.468306 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a1f66-3c11-4a6d-bee2-8a29a1056e76 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:32.479083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a1f66-3c11-4a6d-bee2-8a29a1056e76 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:32.484056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe7b2e-c45b-4260-ad0a-1cb422349930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:32.484995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe7b2e-c45b-4260-ad0a-1cb422349930 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:32.485254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe7b2e-c45b-4260-ad0a-1cb422349930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:32.486419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe7b2e-c45b-4260-ad0a-1cb422349930 admin admin] PATH_INFO: `/v3/projects/a7cf89df79d64153a73a5ec4d49f376f/users/5a7c1a5f674c4dfbb969ac37c5102408/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:32.512827 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe7b2e-c45b-4260-ad0a-1cb422349930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7cf89df79d64153a73a5ec4d49f376f, user_id=5a7c1a5f674c4dfbb969ac37c5102408, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:32.512827 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe7b2e-c45b-4260-ad0a-1cb422349930 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:32.531727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1110/2218] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:32 2022] PUT /identity/v3/projects/a7cf89df79d64153a73a5ec4d49f376f/users/5a7c1a5f674c4dfbb969ac37c5102408/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:32.537398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a42564-c117-4d35-a2ec-0f507a959597 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:32.537691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a42564-c117-4d35-a2ec-0f507a959597 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:32.537968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a42564-c117-4d35-a2ec-0f507a959597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:32.559927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-08a42564-c117-4d35-a2ec-0f507a959597 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:32.561164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-08a42564-c117-4d35-a2ec-0f507a959597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:32.573603 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:32.636330 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1109/2219] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:32.647015 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe7b2e-c45b-4260-ad0a-1cb422349930 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:32.649474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-12fe7b2e-c45b-4260-ad0a-1cb422349930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:32.709341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:32.716149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe7b2e-c45b-4260-ad0a-1cb422349930 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:32.720962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e00f8b-bd60-4415-a433-97bec21add5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:32.721805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e00f8b-bd60-4415-a433-97bec21add5e service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:32.722314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e00f8b-bd60-4415-a433-97bec21add5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:32.722602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e00f8b-bd60-4415-a433-97bec21add5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:32.732540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e00f8b-bd60-4415-a433-97bec21add5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:32.734006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e00f8b-bd60-4415-a433-97bec21add5e service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:32.744992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1111/2220] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:33.477388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a42564-c117-4d35-a2ec-0f507a959597 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:33.479559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-08a42564-c117-4d35-a2ec-0f507a959597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:33.491858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a42564-c117-4d35-a2ec-0f507a959597 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:33.496750 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:33.497895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:33.498189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:33.498476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] PATH_INFO: `/v3/users/047e86053d2c4f41b1f1680a7b259995` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:33.499824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=047e86053d2c4f41b1f1680a7b259995)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:33.501923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:33.572249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '047e86053d2c4f41b1f1680a7b259995'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:33.583479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '047e86053d2c4f41b1f1680a7b259995'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:33.584114 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '047e86053d2c4f41b1f1680a7b259995'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:33.595027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '047e86053d2c4f41b1f1680a7b259995'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:33.596683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '047e86053d2c4f41b1f1680a7b259995'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:33.623333 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1110/2221] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:33 2022] DELETE /identity/v3/users/047e86053d2c4f41b1f1680a7b259995 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:33.625322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a1ee4-16f4-4c97-805c-60a3de83f512 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:33.626215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a1ee4-16f4-4c97-805c-60a3de83f512 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:33.626643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a1ee4-16f4-4c97-805c-60a3de83f512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:33.648421 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-851a1ee4-16f4-4c97-805c-60a3de83f512 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:33.650433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-851a1ee4-16f4-4c97-805c-60a3de83f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:33.699534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:33.702074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:33.752599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1112/2222] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:09:33.764365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a1ee4-16f4-4c97-805c-60a3de83f512 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:33.766224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-851a1ee4-16f4-4c97-805c-60a3de83f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:33.824949 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e2014-c725-474d-86a5-12aae52081a3 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:33.830940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:33.831187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:33.831491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:33.831821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] PATH_INFO: `/v3/users/99127dff61834187834bff45137b5747` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:33.833259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=99127dff61834187834bff45137b5747)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:33.835825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:33.857363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a1ee4-16f4-4c97-805c-60a3de83f512 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:33.863031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff69d9b-d4c2-4e9c-9719-ed8686e59dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:33.863981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff69d9b-d4c2-4e9c-9719-ed8686e59dde service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:33.864450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff69d9b-d4c2-4e9c-9719-ed8686e59dde service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:33.864925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff69d9b-d4c2-4e9c-9719-ed8686e59dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:33.879766 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:33.881252 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff69d9b-d4c2-4e9c-9719-ed8686e59dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:33.884197 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff69d9b-d4c2-4e9c-9719-ed8686e59dde service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:33.904684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1113/2223] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:33.926174 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99127dff61834187834bff45137b5747'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:33.938082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99127dff61834187834bff45137b5747'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:33.939182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99127dff61834187834bff45137b5747'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:33.952295 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99127dff61834187834bff45137b5747'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:33.954463 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99127dff61834187834bff45137b5747'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:34.002794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1111/2224] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:33 2022] DELETE /identity/v3/users/99127dff61834187834bff45137b5747 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:34.007838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d883e9f-f266-4362-86c2-ea798c1f465b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:34.008134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d883e9f-f266-4362-86c2-ea798c1f465b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:34.008416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d883e9f-f266-4362-86c2-ea798c1f465b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:34.031966 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2d883e9f-f266-4362-86c2-ea798c1f465b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:34.033302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2d883e9f-f266-4362-86c2-ea798c1f465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:34.175038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1114/2225] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:34.200924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:34.203413 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:34.310393 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcaccb-9cab-4bfd-8a0b-0268d3d1c5ea admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:34.315426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73251b2-8794-4fdf-938a-54328d791d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:34.316385 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73251b2-8794-4fdf-938a-54328d791d57 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:34.316906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73251b2-8794-4fdf-938a-54328d791d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:34.317260 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73251b2-8794-4fdf-938a-54328d791d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:34.332149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73251b2-8794-4fdf-938a-54328d791d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:34.333802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73251b2-8794-4fdf-938a-54328d791d57 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:34.342643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d883e9f-f266-4362-86c2-ea798c1f465b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:34.344418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2d883e9f-f266-4362-86c2-ea798c1f465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:34.364192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1112/2226] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:34.365934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d883e9f-f266-4362-86c2-ea798c1f465b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:34.374194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:34.375094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:34.375399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:34.375687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] PATH_INFO: `/v3/projects/6e18d83c0c834079bd6520c2eae75dd6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:34.377277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e18d83c0c834079bd6520c2eae75dd6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:34.378995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:34.459064 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:34.508996 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e18d83c0c834079bd6520c2eae75dd6', 'request_id': 'req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:34.522205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e18d83c0c834079bd6520c2eae75dd6', 'request_id': 'req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:34.535313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e18d83c0c834079bd6520c2eae75dd6', 'request_id': 'req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:34.570806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] The token cache is being invalidate because project 6e18d83c0c834079bd6520c2eae75dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:34.571451 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:34.572304 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:34.575026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1115/2227] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:34 2022] DELETE /identity/v3/projects/6e18d83c0c834079bd6520c2eae75dd6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:09:34.579573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73251b2-8794-4fdf-938a-54328d791d57 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:34.603349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73251b2-8794-4fdf-938a-54328d791d57 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:34.695221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:34.697686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:34.723453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68e055-b25b-42eb-8328-d45642e2bf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:34.724625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68e055-b25b-42eb-8328-d45642e2bf58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:34.724824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68e055-b25b-42eb-8328-d45642e2bf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:34.725180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68e055-b25b-42eb-8328-d45642e2bf58 admin admin] PATH_INFO: `/v3/users/90d53d0ec2204f73b96a2309138786ea` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:34.727370 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68e055-b25b-42eb-8328-d45642e2bf58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90d53d0ec2204f73b96a2309138786ea)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:34.730973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68e055-b25b-42eb-8328-d45642e2bf58 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:34.812651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-eb68e055-b25b-42eb-8328-d45642e2bf58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90d53d0ec2204f73b96a2309138786ea'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:34.816302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:34.828189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-eb68e055-b25b-42eb-8328-d45642e2bf58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90d53d0ec2204f73b96a2309138786ea'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:34.829850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-eb68e055-b25b-42eb-8328-d45642e2bf58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90d53d0ec2204f73b96a2309138786ea'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:34.842931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-eb68e055-b25b-42eb-8328-d45642e2bf58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90d53d0ec2204f73b96a2309138786ea'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:34.843866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-eb68e055-b25b-42eb-8328-d45642e2bf58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90d53d0ec2204f73b96a2309138786ea'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:34.844637 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f93b07-4c2d-49e8-a64d-e6ed74f31663 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:34.849727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:34.850683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:34.851806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:34.852109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20 admin admin] PATH_INFO: `/v3/projects/6d40780d34214bc6974bcb658be6b3d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:34.853568 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d40780d34214bc6974bcb658be6b3d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:34.855871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:34.875064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1113/2228] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:34 2022] DELETE /identity/v3/users/90d53d0ec2204f73b96a2309138786ea => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:34.978600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d40780d34214bc6974bcb658be6b3d2', 'request_id': 'req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.004684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d40780d34214bc6974bcb658be6b3d2', 'request_id': 'req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.031079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d40780d34214bc6974bcb658be6b3d2', 'request_id': 'req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.051973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20 admin admin] The token cache is being invalidate because project 6d40780d34214bc6974bcb658be6b3d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:35.052511 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.053755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9fe1a0af-80bc-49e2-acbd-f7f2b4a06b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.057332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1116/2229] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:34 2022] DELETE /identity/v3/projects/6d40780d34214bc6974bcb658be6b3d2 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:35.065005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8870-41a7-4906-9bea-998925979f74 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:35.065641 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8870-41a7-4906-9bea-998925979f74 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:35.066110 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8870-41a7-4906-9bea-998925979f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:35.121077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-932b8870-41a7-4906-9bea-998925979f74 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:35.123494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-932b8870-41a7-4906-9bea-998925979f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:35.244437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1114/2230] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:35.255473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d6cb7-6cdc-4eea-8aba-37bc9d348d0b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:35.256008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d6cb7-6cdc-4eea-8aba-37bc9d348d0b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:35.256191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d6cb7-6cdc-4eea-8aba-37bc9d348d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:35.280981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8d8d6cb7-6cdc-4eea-8aba-37bc9d348d0b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:35.282369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8d8d6cb7-6cdc-4eea-8aba-37bc9d348d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:35.291534 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1117/2231] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:35.296192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b8870-41a7-4906-9bea-998925979f74 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:35.316225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b8870-41a7-4906-9bea-998925979f74 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:35.327310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ef99c-9a0a-4892-86e5-c92d409141dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:35.328037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ef99c-9a0a-4892-86e5-c92d409141dc admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:35.328320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ef99c-9a0a-4892-86e5-c92d409141dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:35.328593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ef99c-9a0a-4892-86e5-c92d409141dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:35.329330 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ef99c-9a0a-4892-86e5-c92d409141dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:09:35.329649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ef99c-9a0a-4892-86e5-c92d409141dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:35.331272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ef99c-9a0a-4892-86e5-c92d409141dc admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:35.359348 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1115/2232] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:09:35.364030 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d6cb7-6cdc-4eea-8aba-37bc9d348d0b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:35.381786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d6cb7-6cdc-4eea-8aba-37bc9d348d0b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:35.390467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d0ba0-f2b3-4388-b2f1-dd1d5a97a72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:35.391705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d0ba0-f2b3-4388-b2f1-dd1d5a97a72e admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:35.394700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d0ba0-f2b3-4388-b2f1-dd1d5a97a72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:35.394700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d0ba0-f2b3-4388-b2f1-dd1d5a97a72e admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:35.394700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d0ba0-f2b3-4388-b2f1-dd1d5a97a72e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:35.397318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ef99c-9a0a-4892-86e5-c92d409141dc admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:35.398957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d0ba0-f2b3-4388-b2f1-dd1d5a97a72e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:35.400748 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ec8ef99c-9a0a-4892-86e5-c92d409141dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:35.440090 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ef99c-9a0a-4892-86e5-c92d409141dc admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:35.447210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:35.448884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:35.449383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:35.449885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] PATH_INFO: `/v3/projects/ca21978c2f044cf9b2f21e0fc058a52a` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:35.452074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca21978c2f044cf9b2f21e0fc058a52a)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:35.454479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:35.468067 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1118/2233] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:09:35 2022] POST /identity/v3/projects => generated 391 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:35.471908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d0ba0-f2b3-4388-b2f1-dd1d5a97a72e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:35.501233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:35.536137 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca21978c2f044cf9b2f21e0fc058a52a', 'request_id': 'req-5c79c7c5-27d4-4338-b964-51e0bf5482c5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.544891 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca21978c2f044cf9b2f21e0fc058a52a', 'request_id': 'req-5c79c7c5-27d4-4338-b964-51e0bf5482c5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.553239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca21978c2f044cf9b2f21e0fc058a52a', 'request_id': 'req-5c79c7c5-27d4-4338-b964-51e0bf5482c5'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.576343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] The token cache is being invalidate because project ca21978c2f044cf9b2f21e0fc058a52a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:35.576794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.577807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.581199 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1116/2234] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:35 2022] DELETE /identity/v3/projects/ca21978c2f044cf9b2f21e0fc058a52a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:35.586979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:35.589729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d0ba0-f2b3-4388-b2f1-dd1d5a97a72e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:35.592033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:35.687514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:35.720334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ad278-d21e-4c7d-9edf-f56d79301045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:35.721719 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ad278-d21e-4c7d-9edf-f56d79301045 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:35.722238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ad278-d21e-4c7d-9edf-f56d79301045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:35.722589 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ad278-d21e-4c7d-9edf-f56d79301045 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:35.723955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ad278-d21e-4c7d-9edf-f56d79301045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:35.727342 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ad278-d21e-4c7d-9edf-f56d79301045 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:35.736427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79c7c5-27d4-4338-b964-51e0bf5482c5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:35.744536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bcacc-beec-4f50-be9c-39f0acf94095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:35.745565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bcacc-beec-4f50-be9c-39f0acf94095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:35.745936 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bcacc-beec-4f50-be9c-39f0acf94095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:35.746299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bcacc-beec-4f50-be9c-39f0acf94095 admin admin] PATH_INFO: `/v3/users/662920c0c920494796cc98de7e5fe334` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:35.748683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bcacc-beec-4f50-be9c-39f0acf94095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=662920c0c920494796cc98de7e5fe334)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:35.764133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bcacc-beec-4f50-be9c-39f0acf94095 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:35.768368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1119/2235] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:09:35 2022] POST /identity/v3/users => generated 296 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:09:35.779190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ad278-d21e-4c7d-9edf-f56d79301045 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:35.792644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cf0ad278-d21e-4c7d-9edf-f56d79301045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:35.823201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ad278-d21e-4c7d-9edf-f56d79301045 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:35.832848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59326b68-4a36-4387-81e3-833ee5106203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:35.834066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59326b68-4a36-4387-81e3-833ee5106203 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:35.835085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59326b68-4a36-4387-81e3-833ee5106203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:35.836583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59326b68-4a36-4387-81e3-833ee5106203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:35.838424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59326b68-4a36-4387-81e3-833ee5106203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:35.842154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59326b68-4a36-4387-81e3-833ee5106203 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:35.888134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1120/2236] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:35.891991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59326b68-4a36-4387-81e3-833ee5106203 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:35.896006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4d2bcacc-beec-4f50-be9c-39f0acf94095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '662920c0c920494796cc98de7e5fe334'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.915731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4d2bcacc-beec-4f50-be9c-39f0acf94095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '662920c0c920494796cc98de7e5fe334'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.916627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4d2bcacc-beec-4f50-be9c-39f0acf94095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '662920c0c920494796cc98de7e5fe334'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.922124 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59326b68-4a36-4387-81e3-833ee5106203 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:35.935058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4d2bcacc-beec-4f50-be9c-39f0acf94095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '662920c0c920494796cc98de7e5fe334'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.937012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4d2bcacc-beec-4f50-be9c-39f0acf94095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '662920c0c920494796cc98de7e5fe334'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:35.938034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd9841-c43b-4c91-99a0-5d37f7851c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:35.940497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd9841-c43b-4c91-99a0-5d37f7851c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:35.940905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd9841-c43b-4c91-99a0-5d37f7851c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:35.943365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd9841-c43b-4c91-99a0-5d37f7851c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:35.946501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd9841-c43b-4c91-99a0-5d37f7851c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:35.948912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd9841-c43b-4c91-99a0-5d37f7851c3c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:35.978745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1121/2237] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:09:35.979931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd9841-c43b-4c91-99a0-5d37f7851c3c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:35.983871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c8bd9841-c43b-4c91-99a0-5d37f7851c3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:35.987422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1117/2238] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:09:35 2022] DELETE /identity/v3/users/662920c0c920494796cc98de7e5fe334 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:36.003559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd9841-c43b-4c91-99a0-5d37f7851c3c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:36.123827 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582ff77-540c-4bb1-b4e5-423f1a70e605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:36.126170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582ff77-540c-4bb1-b4e5-423f1a70e605 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:36.128862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582ff77-540c-4bb1-b4e5-423f1a70e605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:36.129763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582ff77-540c-4bb1-b4e5-423f1a70e605 admin admin] PATH_INFO: `/v3/projects/ff13783fe5b44537ae749607d0fe9a14/users/16ec4fb3fe3840d7b7f754e168cb1e75/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:36.166995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582ff77-540c-4bb1-b4e5-423f1a70e605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff13783fe5b44537ae749607d0fe9a14, user_id=16ec4fb3fe3840d7b7f754e168cb1e75, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:36.171946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582ff77-540c-4bb1-b4e5-423f1a70e605 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:36.207577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1122/2239] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:09:35 2022] PUT /identity/v3/projects/ff13783fe5b44537ae749607d0fe9a14/users/16ec4fb3fe3840d7b7f754e168cb1e75/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:36.219447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cf8fb-24e4-4815-8810-a50b1c62d642 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:36.219757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cf8fb-24e4-4815-8810-a50b1c62d642 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:36.220048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cf8fb-24e4-4815-8810-a50b1c62d642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:36.277359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-821cf8fb-24e4-4815-8810-a50b1c62d642 None None] MFA Rules not processed for user `16ec4fb3fe3840d7b7f754e168cb1e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:36.279699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-821cf8fb-24e4-4815-8810-a50b1c62d642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:36.408159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1118/2240] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:36.421465 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582ff77-540c-4bb1-b4e5-423f1a70e605 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:36.424280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0582ff77-540c-4bb1-b4e5-423f1a70e605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:36.472932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:36.493031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cf8fb-24e4-4815-8810-a50b1c62d642 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:36.496205 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-821cf8fb-24e4-4815-8810-a50b1c62d642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:36.594936 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582ff77-540c-4bb1-b4e5-423f1a70e605 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:36.599133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de4465-4cba-456b-9ae7-25f8fae17b0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:36.599851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de4465-4cba-456b-9ae7-25f8fae17b0c service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:36.600144 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de4465-4cba-456b-9ae7-25f8fae17b0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:36.600445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de4465-4cba-456b-9ae7-25f8fae17b0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:36.610906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de4465-4cba-456b-9ae7-25f8fae17b0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:36.614203 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de4465-4cba-456b-9ae7-25f8fae17b0c service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:36.615017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:36.632290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1123/2241] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:36.641804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cf8fb-24e4-4815-8810-a50b1c62d642 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:36.664936 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:36.667231 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:36.667859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:36.668663 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] PATH_INFO: `/v3/projects/768738017c28424aab490f8d79d9dace` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:36.671780 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=768738017c28424aab490f8d79d9dace)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:36.674035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:36.757507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '768738017c28424aab490f8d79d9dace', 'request_id': 'req-4a57a7e2-013e-4158-a508-a7b1196a8aab'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:36.765603 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '768738017c28424aab490f8d79d9dace', 'request_id': 'req-4a57a7e2-013e-4158-a508-a7b1196a8aab'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:36.775741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '768738017c28424aab490f8d79d9dace', 'request_id': 'req-4a57a7e2-013e-4158-a508-a7b1196a8aab'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:36.801722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] The token cache is being invalidate because project 768738017c28424aab490f8d79d9dace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:09:36.802178 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:36.802932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:09:36.807863 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1119/2242] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:09:36 2022] DELETE /identity/v3/projects/768738017c28424aab490f8d79d9dace => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:09:38.684865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e365e7-6cd7-441a-ab79-be5385a89f78 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:38.685471 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e365e7-6cd7-441a-ab79-be5385a89f78 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:38.685471 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e365e7-6cd7-441a-ab79-be5385a89f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:38.731435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-86e365e7-6cd7-441a-ab79-be5385a89f78 None None] MFA Rules not processed for user `5a7c1a5f674c4dfbb969ac37c5102408`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:38.734220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-86e365e7-6cd7-441a-ab79-be5385a89f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:38.859586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1124/2243] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:38.882147 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:38.884423 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:38.986266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57a7e2-013e-4158-a508-a7b1196a8aab admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:38.992744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff5d64-9d10-45c5-a2e8-9e8faf4a46a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:38.993420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff5d64-9d10-45c5-a2e8-9e8faf4a46a3 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:38.993702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff5d64-9d10-45c5-a2e8-9e8faf4a46a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:38.993979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff5d64-9d10-45c5-a2e8-9e8faf4a46a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:39.003011 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff5d64-9d10-45c5-a2e8-9e8faf4a46a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:39.005246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff5d64-9d10-45c5-a2e8-9e8faf4a46a3 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:39.015605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1120/2244] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:09:40.590497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dda71-fbf0-4bc3-8558-1bccd2a9b240 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:40.591182 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dda71-fbf0-4bc3-8558-1bccd2a9b240 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:40.591509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dda71-fbf0-4bc3-8558-1bccd2a9b240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:40.624456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-713dda71-fbf0-4bc3-8558-1bccd2a9b240 None None] MFA Rules not processed for user `5a7c1a5f674c4dfbb969ac37c5102408`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:40.626864 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-713dda71-fbf0-4bc3-8558-1bccd2a9b240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:40.651375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1125/2245] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:09:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:49.011187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=64,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:50.609987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=54 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:10.279125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff5d64-9d10-45c5-a2e8-9e8faf4a46a3 service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:10.281175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8aff5d64-9d10-45c5-a2e8-9e8faf4a46a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:10.369188 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff5d64-9d10-45c5-a2e8-9e8faf4a46a3 service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:10.374810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:10.375762 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:10.376052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:10.376335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] PATH_INFO: `/v3/users/16ec4fb3fe3840d7b7f754e168cb1e75` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:10.377660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=16ec4fb3fe3840d7b7f754e168cb1e75)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:10.379873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:10.442313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16ec4fb3fe3840d7b7f754e168cb1e75'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:10.453915 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16ec4fb3fe3840d7b7f754e168cb1e75'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:10.454810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16ec4fb3fe3840d7b7f754e168cb1e75'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:10.469070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16ec4fb3fe3840d7b7f754e168cb1e75'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:10.470243 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16ec4fb3fe3840d7b7f754e168cb1e75'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:10.507296 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1121/2246] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:10:10 2022] DELETE /identity/v3/users/16ec4fb3fe3840d7b7f754e168cb1e75 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:10:10.511914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea53681-cad1-4eb4-bd63-75e57f952d51 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:10.512654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea53681-cad1-4eb4-bd63-75e57f952d51 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:10.512654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea53681-cad1-4eb4-bd63-75e57f952d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:10.536014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-aea53681-cad1-4eb4-bd63-75e57f952d51 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:10.538190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-aea53681-cad1-4eb4-bd63-75e57f952d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:10.604480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1126/2247] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:10.614785 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:10.617033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:10.681086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8fb86-0e3a-4b7c-97f4-61ff0238fd1d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:10.687814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b110a-036e-41f0-97e4-0d003b3bae2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:10.688492 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b110a-036e-41f0-97e4-0d003b3bae2c service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:10.688781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b110a-036e-41f0-97e4-0d003b3bae2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:10.689063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b110a-036e-41f0-97e4-0d003b3bae2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:10.700118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b110a-036e-41f0-97e4-0d003b3bae2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:10.701629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b110a-036e-41f0-97e4-0d003b3bae2c service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:10.712322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1122/2248] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:11.044751 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea53681-cad1-4eb4-bd63-75e57f952d51 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:11.047428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-aea53681-cad1-4eb4-bd63-75e57f952d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:11.065464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea53681-cad1-4eb4-bd63-75e57f952d51 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:11.070868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:11.071670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:11.072234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:11.072234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] PATH_INFO: `/v3/projects/ff13783fe5b44537ae749607d0fe9a14` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:11.073536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff13783fe5b44537ae749607d0fe9a14)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:11.075693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:11.135684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff13783fe5b44537ae749607d0fe9a14', 'request_id': 'req-e1bec796-ff7a-4705-a938-e0d319bfee68'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:11.149260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff13783fe5b44537ae749607d0fe9a14', 'request_id': 'req-e1bec796-ff7a-4705-a938-e0d319bfee68'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:11.159921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff13783fe5b44537ae749607d0fe9a14', 'request_id': 'req-e1bec796-ff7a-4705-a938-e0d319bfee68'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:11.176369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] The token cache is being invalidate because project ff13783fe5b44537ae749607d0fe9a14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:10:11.176834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:11.177499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:11.179986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1127/2249] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:10:11 2022] DELETE /identity/v3/projects/ff13783fe5b44537ae749607d0fe9a14 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:10:20.707660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:21.172920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:21.541519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b110a-036e-41f0-97e4-0d003b3bae2c service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:21.543912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f34b110a-036e-41f0-97e4-0d003b3bae2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:21.653165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b110a-036e-41f0-97e4-0d003b3bae2c service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:21.733765 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04a79b-2381-48fd-8e9c-a9513d406e00 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:21.734619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04a79b-2381-48fd-8e9c-a9513d406e00 service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:21.734853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04a79b-2381-48fd-8e9c-a9513d406e00 service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:21.735538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04a79b-2381-48fd-8e9c-a9513d406e00 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:21.738372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8a04a79b-2381-48fd-8e9c-a9513d406e00 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:21.751799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04a79b-2381-48fd-8e9c-a9513d406e00 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:21.753455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04a79b-2381-48fd-8e9c-a9513d406e00 service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:21.766801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1123/2250] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:22.520941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:22.522494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:22.610812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bec796-ff7a-4705-a938-e0d319bfee68 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:22.616688 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:22.617611 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:22.617890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:22.618174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] PATH_INFO: `/v3/users/1f6f5ea786e743b69b9311e49af302d6` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:22.619667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f6f5ea786e743b69b9311e49af302d6)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:22.621643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:22.687258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f6f5ea786e743b69b9311e49af302d6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:22.687977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f6f5ea786e743b69b9311e49af302d6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:22.695984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f6f5ea786e743b69b9311e49af302d6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:22.705077 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f6f5ea786e743b69b9311e49af302d6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:22.705972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f6f5ea786e743b69b9311e49af302d6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:22.728796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1128/2251] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:10:22 2022] DELETE /identity/v3/users/1f6f5ea786e743b69b9311e49af302d6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:10:22.733695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b9410-1fbf-48ef-b08f-8997c44ddc38 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:22.733932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b9410-1fbf-48ef-b08f-8997c44ddc38 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:22.734495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b9410-1fbf-48ef-b08f-8997c44ddc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:22.754757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-779b9410-1fbf-48ef-b08f-8997c44ddc38 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:22.756795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-779b9410-1fbf-48ef-b08f-8997c44ddc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:22.824579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1124/2252] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:10:22.834874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:22.836461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:22.907190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b918064-b974-4a71-a988-ca960c9eddd4 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:22.910919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1975ed-d3ec-4455-bccd-47212c82cae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:22.911607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1975ed-d3ec-4455-bccd-47212c82cae2 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:22.911913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1975ed-d3ec-4455-bccd-47212c82cae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:22.912722 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1975ed-d3ec-4455-bccd-47212c82cae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:22.923184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1975ed-d3ec-4455-bccd-47212c82cae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:22.924755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1975ed-d3ec-4455-bccd-47212c82cae2 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:22.935521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1129/2253] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:23.268530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b9410-1fbf-48ef-b08f-8997c44ddc38 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:23.270685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-779b9410-1fbf-48ef-b08f-8997c44ddc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:23.293720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b9410-1fbf-48ef-b08f-8997c44ddc38 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:23.299136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff9818-a203-4d36-a2d2-a65e9a6f48ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:23.300060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff9818-a203-4d36-a2d2-a65e9a6f48ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:23.300342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff9818-a203-4d36-a2d2-a65e9a6f48ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:23.300616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff9818-a203-4d36-a2d2-a65e9a6f48ac admin admin] PATH_INFO: `/v3/projects/5ce5cf1763614953b1d376be9e77f2fd` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:23.301863 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff9818-a203-4d36-a2d2-a65e9a6f48ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ce5cf1763614953b1d376be9e77f2fd)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:23.304744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff9818-a203-4d36-a2d2-a65e9a6f48ac admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:23.351584 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:23.373379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-afff9818-a203-4d36-a2d2-a65e9a6f48ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ce5cf1763614953b1d376be9e77f2fd', 'request_id': 'req-afff9818-a203-4d36-a2d2-a65e9a6f48ac'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:23.381798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-afff9818-a203-4d36-a2d2-a65e9a6f48ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ce5cf1763614953b1d376be9e77f2fd', 'request_id': 'req-afff9818-a203-4d36-a2d2-a65e9a6f48ac'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:23.393443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-afff9818-a203-4d36-a2d2-a65e9a6f48ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ce5cf1763614953b1d376be9e77f2fd', 'request_id': 'req-afff9818-a203-4d36-a2d2-a65e9a6f48ac'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:23.409676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-afff9818-a203-4d36-a2d2-a65e9a6f48ac admin admin] The token cache is being invalidate because project 5ce5cf1763614953b1d376be9e77f2fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:10:23.410054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-afff9818-a203-4d36-a2d2-a65e9a6f48ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:23.410813 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-afff9818-a203-4d36-a2d2-a65e9a6f48ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:23.412658 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1125/2254] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:10:23 2022] DELETE /identity/v3/projects/5ce5cf1763614953b1d376be9e77f2fd => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:10:23.417879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7356c3b-435d-4375-ae24-f92cced830ae None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:23.418305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7356c3b-435d-4375-ae24-f92cced830ae None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:23.418675 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7356c3b-435d-4375-ae24-f92cced830ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:23.438498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f7356c3b-435d-4375-ae24-f92cced830ae None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:23.439918 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f7356c3b-435d-4375-ae24-f92cced830ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:23.465260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:23.530546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1130/2255] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:10:23.544325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42587c42-3e97-466f-a9c6-cccc44809e08 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:23.545002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42587c42-3e97-466f-a9c6-cccc44809e08 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:23.545641 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42587c42-3e97-466f-a9c6-cccc44809e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:23.581523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-42587c42-3e97-466f-a9c6-cccc44809e08 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:23.582498 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-42587c42-3e97-466f-a9c6-cccc44809e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:23.592196 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1126/2256] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:23.597231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7356c3b-435d-4375-ae24-f92cced830ae None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:23.608175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7356c3b-435d-4375-ae24-f92cced830ae None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:23.612983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c0fd5-c02b-4771-aa16-54a3a9b140bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:23.613670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c0fd5-c02b-4771-aa16-54a3a9b140bf admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:23.613959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c0fd5-c02b-4771-aa16-54a3a9b140bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:23.614246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c0fd5-c02b-4771-aa16-54a3a9b140bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:23.615008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c0fd5-c02b-4771-aa16-54a3a9b140bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:10:23.615951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c0fd5-c02b-4771-aa16-54a3a9b140bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:23.617601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c0fd5-c02b-4771-aa16-54a3a9b140bf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:23.630180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1131/2257] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:10:23.635547 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42587c42-3e97-466f-a9c6-cccc44809e08 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:23.647272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42587c42-3e97-466f-a9c6-cccc44809e08 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:23.653714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48aa7b9-fae6-4854-9439-ea67304027b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:23.654909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48aa7b9-fae6-4854-9439-ea67304027b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:23.655684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48aa7b9-fae6-4854-9439-ea67304027b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:23.656216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48aa7b9-fae6-4854-9439-ea67304027b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:23.657659 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48aa7b9-fae6-4854-9439-ea67304027b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:23.661970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48aa7b9-fae6-4854-9439-ea67304027b2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:23.712139 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1127/2258] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:10:23 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:10:23.715296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c0fd5-c02b-4771-aa16-54a3a9b140bf admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:23.733567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c0fd5-c02b-4771-aa16-54a3a9b140bf admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:23.812684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e9a27-2d62-4562-bb06-81977e8f67f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:23.813406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e9a27-2d62-4562-bb06-81977e8f67f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:23.814295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e9a27-2d62-4562-bb06-81977e8f67f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:23.814295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e9a27-2d62-4562-bb06-81977e8f67f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:23.815157 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e9a27-2d62-4562-bb06-81977e8f67f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:23.817275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e9a27-2d62-4562-bb06-81977e8f67f8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:23.847333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1132/2259] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:10:23 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:10:23.849167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48aa7b9-fae6-4854-9439-ea67304027b2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:23.862777 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48aa7b9-fae6-4854-9439-ea67304027b2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:23.874687 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedd9c5-1528-4d8b-8d4a-634212ecdc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:23.876121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedd9c5-1528-4d8b-8d4a-634212ecdc0a admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:23.877199 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedd9c5-1528-4d8b-8d4a-634212ecdc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:23.877780 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedd9c5-1528-4d8b-8d4a-634212ecdc0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:23.880052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedd9c5-1528-4d8b-8d4a-634212ecdc0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:23.885037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedd9c5-1528-4d8b-8d4a-634212ecdc0a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:23.908249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1128/2260] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:10:23.913781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e9a27-2d62-4562-bb06-81977e8f67f8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:23.926846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e9a27-2d62-4562-bb06-81977e8f67f8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:23.933713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bf7e7-ff7d-482b-a58d-3026d69245f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:23.934508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bf7e7-ff7d-482b-a58d-3026d69245f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:23.934793 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bf7e7-ff7d-482b-a58d-3026d69245f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:23.935068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bf7e7-ff7d-482b-a58d-3026d69245f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:23.935826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bf7e7-ff7d-482b-a58d-3026d69245f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:23.937420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bf7e7-ff7d-482b-a58d-3026d69245f9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:23.949342 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1133/2261] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:10:23.956332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedd9c5-1528-4d8b-8d4a-634212ecdc0a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:23.971979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedd9c5-1528-4d8b-8d4a-634212ecdc0a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:23.980322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705bf52-f1a3-4329-b0e7-45a55857c343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:23.982773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705bf52-f1a3-4329-b0e7-45a55857c343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:23.983485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705bf52-f1a3-4329-b0e7-45a55857c343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:23.983823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705bf52-f1a3-4329-b0e7-45a55857c343 admin admin] PATH_INFO: `/v3/projects/4dff93621a614bf4b6c7ec383e42019a/users/5fb87c0f30b04869bbeada163de623c0/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:24.001675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705bf52-f1a3-4329-b0e7-45a55857c343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dff93621a614bf4b6c7ec383e42019a, user_id=5fb87c0f30b04869bbeada163de623c0, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:24.004480 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705bf52-f1a3-4329-b0e7-45a55857c343 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:24.023640 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1129/2262] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:10:23 2022] PUT /identity/v3/projects/4dff93621a614bf4b6c7ec383e42019a/users/5fb87c0f30b04869bbeada163de623c0/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:10:24.043145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1fa45-9ac1-4b39-a7d3-a9da410c0a30 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:24.043809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1fa45-9ac1-4b39-a7d3-a9da410c0a30 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:24.044372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1fa45-9ac1-4b39-a7d3-a9da410c0a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:24.080580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c8c1fa45-9ac1-4b39-a7d3-a9da410c0a30 None None] MFA Rules not processed for user `5fb87c0f30b04869bbeada163de623c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:24.081706 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c8c1fa45-9ac1-4b39-a7d3-a9da410c0a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:24.197782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1134/2263] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:10:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:24.216130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705bf52-f1a3-4329-b0e7-45a55857c343 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:24.218820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4705bf52-f1a3-4329-b0e7-45a55857c343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:24.298865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705bf52-f1a3-4329-b0e7-45a55857c343 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:24.302595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6117e8a-0312-45f0-af00-c0ddef74e495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:24.303266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6117e8a-0312-45f0-af00-c0ddef74e495 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:24.303582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6117e8a-0312-45f0-af00-c0ddef74e495 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:24.304132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6117e8a-0312-45f0-af00-c0ddef74e495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:24.314414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6117e8a-0312-45f0-af00-c0ddef74e495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:24.316475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6117e8a-0312-45f0-af00-c0ddef74e495 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:24.327224 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1130/2264] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:24.359263 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1fa45-9ac1-4b39-a7d3-a9da410c0a30 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:24.392222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:24.457481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1fa45-9ac1-4b39-a7d3-a9da410c0a30 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:24.462319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:24.463241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:24.463553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:24.464364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] PATH_INFO: `/v3/users/5fb87c0f30b04869bbeada163de623c0` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:24.465079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb87c0f30b04869bbeada163de623c0)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:24.467012 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:24.546546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb87c0f30b04869bbeada163de623c0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:24.547344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb87c0f30b04869bbeada163de623c0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:24.556654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb87c0f30b04869bbeada163de623c0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:24.565526 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb87c0f30b04869bbeada163de623c0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:24.566339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb87c0f30b04869bbeada163de623c0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:24.596617 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1135/2265] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:10:24 2022] DELETE /identity/v3/users/5fb87c0f30b04869bbeada163de623c0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:10:24.601673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be6a84-0014-4870-a5f0-8de44299b4ad None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:24.601979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be6a84-0014-4870-a5f0-8de44299b4ad None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:24.602264 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be6a84-0014-4870-a5f0-8de44299b4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:24.621124 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-71be6a84-0014-4870-a5f0-8de44299b4ad None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:24.622367 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-71be6a84-0014-4870-a5f0-8de44299b4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:24.679588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:24.705613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1131/2266] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:24.713496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:24.716062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:24.799921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fd3ef-6672-463c-a8b0-8ffc718f3b6e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:24.807086 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56e8d1-4e95-4845-adb4-c1c8c24c67a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:24.808289 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56e8d1-4e95-4845-adb4-c1c8c24c67a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:24.808791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56e8d1-4e95-4845-adb4-c1c8c24c67a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:24.809310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56e8d1-4e95-4845-adb4-c1c8c24c67a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:24.822610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56e8d1-4e95-4845-adb4-c1c8c24c67a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:24.824133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56e8d1-4e95-4845-adb4-c1c8c24c67a1 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:24.834969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1136/2267] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:24.951801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f8f58-8936-48f9-aff6-da4973dc8aa1 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:24.952686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f8f58-8936-48f9-aff6-da4973dc8aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:24.952686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f8f58-8936-48f9-aff6-da4973dc8aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:24.972102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-088f8f58-8936-48f9-aff6-da4973dc8aa1 None None] MFA Rules not processed for user `cb80e0c64c3348cfac5ad2c4f82fb606`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:24.973557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-088f8f58-8936-48f9-aff6-da4973dc8aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:25.039198 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1132/2268] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:10:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:25.242315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56e8d1-4e95-4845-adb4-c1c8c24c67a1 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:25.243799 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9c56e8d1-4e95-4845-adb4-c1c8c24c67a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:25.257734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56e8d1-4e95-4845-adb4-c1c8c24c67a1 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:25.262632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:25.263644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:25.263935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:25.264214 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d admin admin] PATH_INFO: `/v3/projects/4dff93621a614bf4b6c7ec383e42019a` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:25.265443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dff93621a614bf4b6c7ec383e42019a)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:25.267422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:25.327609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:25.346372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dff93621a614bf4b6c7ec383e42019a', 'request_id': 'req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:25.358825 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dff93621a614bf4b6c7ec383e42019a', 'request_id': 'req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:25.368979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dff93621a614bf4b6c7ec383e42019a', 'request_id': 'req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:25.385594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d admin admin] The token cache is being invalidate because project 4dff93621a614bf4b6c7ec383e42019a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:10:25.385908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:25.386561 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8aac4a7c-1076-4912-9bf9-35bc88bdba6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:25.388349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1137/2269] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:10:25 2022] DELETE /identity/v3/projects/4dff93621a614bf4b6c7ec383e42019a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:10:25.393428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451290c-9f4a-4397-ad1c-351f60e665d0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:25.393728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451290c-9f4a-4397-ad1c-351f60e665d0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:25.394020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451290c-9f4a-4397-ad1c-351f60e665d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:25.413050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5451290c-9f4a-4397-ad1c-351f60e665d0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:25.414294 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5451290c-9f4a-4397-ad1c-351f60e665d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:25.493131 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1133/2270] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:10:25.501083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657e82-324a-4105-adb0-acdcf8ea3839 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:25.501400 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657e82-324a-4105-adb0-acdcf8ea3839 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:25.501665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657e82-324a-4105-adb0-acdcf8ea3839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:25.522290 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a7657e82-324a-4105-adb0-acdcf8ea3839 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:25.523603 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a7657e82-324a-4105-adb0-acdcf8ea3839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:25.534246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1138/2271] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:25.539124 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451290c-9f4a-4397-ad1c-351f60e665d0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:25.550251 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451290c-9f4a-4397-ad1c-351f60e665d0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:25.556474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1a978-86c5-4034-8d2a-33aac2b53076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:25.557163 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a978-86c5-4034-8d2a-33aac2b53076 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:25.557449 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a978-86c5-4034-8d2a-33aac2b53076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:25.557740 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a978-86c5-4034-8d2a-33aac2b53076 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:25.558466 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1a978-86c5-4034-8d2a-33aac2b53076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:10:25.558773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1a978-86c5-4034-8d2a-33aac2b53076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:25.560443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1a978-86c5-4034-8d2a-33aac2b53076 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:25.573166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1134/2272] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:10:25.577645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7657e82-324a-4105-adb0-acdcf8ea3839 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:25.587360 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7657e82-324a-4105-adb0-acdcf8ea3839 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:25.594174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f8fff-a6dd-4e49-889c-b11a11b70cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:25.594881 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f8fff-a6dd-4e49-889c-b11a11b70cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:25.595188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f8fff-a6dd-4e49-889c-b11a11b70cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:25.595502 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f8fff-a6dd-4e49-889c-b11a11b70cc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:25.596243 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f8fff-a6dd-4e49-889c-b11a11b70cc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:25.598181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f8fff-a6dd-4e49-889c-b11a11b70cc7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:25.629563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1139/2273] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:10:25 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:10:25.634638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1a978-86c5-4034-8d2a-33aac2b53076 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:25.645232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1a978-86c5-4034-8d2a-33aac2b53076 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:25.720705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105431ed-24d5-4db0-a58f-dcd467af5359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:25.721558 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105431ed-24d5-4db0-a58f-dcd467af5359 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:25.721969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105431ed-24d5-4db0-a58f-dcd467af5359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:25.722445 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105431ed-24d5-4db0-a58f-dcd467af5359 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:25.723683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105431ed-24d5-4db0-a58f-dcd467af5359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:25.726926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105431ed-24d5-4db0-a58f-dcd467af5359 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:25.763270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1135/2274] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:10:25 2022] POST /identity/v3/users => generated 300 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:10:25.765440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f8fff-a6dd-4e49-889c-b11a11b70cc7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:25.784291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f8fff-a6dd-4e49-889c-b11a11b70cc7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:25.792331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a01ae0-f362-4c69-8b39-8071253572c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:25.793175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a01ae0-f362-4c69-8b39-8071253572c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:25.793660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a01ae0-f362-4c69-8b39-8071253572c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:25.794060 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a01ae0-f362-4c69-8b39-8071253572c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:25.794978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a01ae0-f362-4c69-8b39-8071253572c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:25.797009 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a01ae0-f362-4c69-8b39-8071253572c1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:25.817116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1140/2275] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:10:25.823429 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105431ed-24d5-4db0-a58f-dcd467af5359 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:25.837447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105431ed-24d5-4db0-a58f-dcd467af5359 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:25.843367 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058d667-4985-4ab5-b4f2-70b9c1d091d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:25.844675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058d667-4985-4ab5-b4f2-70b9c1d091d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:25.845160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058d667-4985-4ab5-b4f2-70b9c1d091d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:25.845435 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058d667-4985-4ab5-b4f2-70b9c1d091d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:25.846213 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058d667-4985-4ab5-b4f2-70b9c1d091d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:25.847931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058d667-4985-4ab5-b4f2-70b9c1d091d9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:25.863090 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1136/2276] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:10:25.867822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a01ae0-f362-4c69-8b39-8071253572c1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:25.880731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a01ae0-f362-4c69-8b39-8071253572c1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:25.888418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26950bd8-f159-472b-bc75-25a61e3b5a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:25.889484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26950bd8-f159-472b-bc75-25a61e3b5a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:25.889820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26950bd8-f159-472b-bc75-25a61e3b5a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:25.890050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26950bd8-f159-472b-bc75-25a61e3b5a38 admin admin] PATH_INFO: `/v3/projects/826d836abf1c4351a17c863dad9a90b5/users/0551d947006041f69b5ce2fe897f6d05/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:25.912591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26950bd8-f159-472b-bc75-25a61e3b5a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=826d836abf1c4351a17c863dad9a90b5, user_id=0551d947006041f69b5ce2fe897f6d05, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:25.920819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26950bd8-f159-472b-bc75-25a61e3b5a38 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:25.954586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1141/2277] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:10:25 2022] PUT /identity/v3/projects/826d836abf1c4351a17c863dad9a90b5/users/0551d947006041f69b5ce2fe897f6d05/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:10:26.032310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c988c1-b53a-4f7f-9991-d8688bf5d095 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:26.032935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c988c1-b53a-4f7f-9991-d8688bf5d095 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:26.032935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c988c1-b53a-4f7f-9991-d8688bf5d095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:26.050514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:26.073315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-39c988c1-b53a-4f7f-9991-d8688bf5d095 None None] MFA Rules not processed for user `0551d947006041f69b5ce2fe897f6d05`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:26.074652 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-39c988c1-b53a-4f7f-9991-d8688bf5d095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:26.158915 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1137/2278] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:10:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:26.174770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26950bd8-f159-472b-bc75-25a61e3b5a38 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:26.176874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-26950bd8-f159-472b-bc75-25a61e3b5a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:26.283145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26950bd8-f159-472b-bc75-25a61e3b5a38 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:26.291363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53e154-74bb-4297-8d3b-8cf76fd2983c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:26.292561 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53e154-74bb-4297-8d3b-8cf76fd2983c service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:26.293082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53e154-74bb-4297-8d3b-8cf76fd2983c service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:26.293608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53e154-74bb-4297-8d3b-8cf76fd2983c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:26.310404 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53e154-74bb-4297-8d3b-8cf76fd2983c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:26.313314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53e154-74bb-4297-8d3b-8cf76fd2983c service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:26.330926 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1142/2279] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:34.614472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c988c1-b53a-4f7f-9991-d8688bf5d095 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:34.677842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c988c1-b53a-4f7f-9991-d8688bf5d095 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:34.682661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:34.683574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:34.683867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:34.684149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] PATH_INFO: `/v3/users/0551d947006041f69b5ce2fe897f6d05` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:34.685348 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0551d947006041f69b5ce2fe897f6d05)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:34.687285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:34.736746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0551d947006041f69b5ce2fe897f6d05'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:34.744262 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0551d947006041f69b5ce2fe897f6d05'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:34.744832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0551d947006041f69b5ce2fe897f6d05'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:34.752017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0551d947006041f69b5ce2fe897f6d05'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:34.752669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0551d947006041f69b5ce2fe897f6d05'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:34.773425 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1138/2280] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:10:34 2022] DELETE /identity/v3/users/0551d947006041f69b5ce2fe897f6d05 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:10:34.777952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095dc69-ff92-4435-b45b-9fe42ddff2cf None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:34.778472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095dc69-ff92-4435-b45b-9fe42ddff2cf None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:34.778588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095dc69-ff92-4435-b45b-9fe42ddff2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:34.797513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c095dc69-ff92-4435-b45b-9fe42ddff2cf None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:10:34.798820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c095dc69-ff92-4435-b45b-9fe42ddff2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:34.860047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1143/2281] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:10:34.870378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:34.871830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:34.890088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:34.934995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34128e2-2991-4f4e-96e5-e1b065fd126d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:34.938776 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e3a11-7a6b-4b39-b298-7ce797e4c46c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:34.939461 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e3a11-7a6b-4b39-b298-7ce797e4c46c service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:34.939753 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e3a11-7a6b-4b39-b298-7ce797e4c46c service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:34.940032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e3a11-7a6b-4b39-b298-7ce797e4c46c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:34.949382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e3a11-7a6b-4b39-b298-7ce797e4c46c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:34.950839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e3a11-7a6b-4b39-b298-7ce797e4c46c service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:34.961350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1139/2282] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:35.233064 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095dc69-ff92-4435-b45b-9fe42ddff2cf None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:35.234520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c095dc69-ff92-4435-b45b-9fe42ddff2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:35.241867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:35.249513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095dc69-ff92-4435-b45b-9fe42ddff2cf None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:35.257821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cecaf-bb6e-488a-93dd-735b2ded480e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:35.259504 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cecaf-bb6e-488a-93dd-735b2ded480e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:35.259504 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cecaf-bb6e-488a-93dd-735b2ded480e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:35.260386 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cecaf-bb6e-488a-93dd-735b2ded480e admin admin] PATH_INFO: `/v3/projects/826d836abf1c4351a17c863dad9a90b5` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:35.261695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cecaf-bb6e-488a-93dd-735b2ded480e admin admin] RBAC: Authorizing `identity:delete_project(project_id=826d836abf1c4351a17c863dad9a90b5)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:35.263685 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cecaf-bb6e-488a-93dd-735b2ded480e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:35.312536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-267cecaf-bb6e-488a-93dd-735b2ded480e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '826d836abf1c4351a17c863dad9a90b5', 'request_id': 'req-267cecaf-bb6e-488a-93dd-735b2ded480e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:35.320352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-267cecaf-bb6e-488a-93dd-735b2ded480e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '826d836abf1c4351a17c863dad9a90b5', 'request_id': 'req-267cecaf-bb6e-488a-93dd-735b2ded480e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:35.326169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-267cecaf-bb6e-488a-93dd-735b2ded480e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '826d836abf1c4351a17c863dad9a90b5', 'request_id': 'req-267cecaf-bb6e-488a-93dd-735b2ded480e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:35.339250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-267cecaf-bb6e-488a-93dd-735b2ded480e admin admin] The token cache is being invalidate because project 826d836abf1c4351a17c863dad9a90b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:10:35.339657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-267cecaf-bb6e-488a-93dd-735b2ded480e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:35.340448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-267cecaf-bb6e-488a-93dd-735b2ded480e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:10:35.341991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1144/2283] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:10:35 2022] DELETE /identity/v3/projects/826d836abf1c4351a17c863dad9a90b5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:10:44.956392 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=19 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:45.336227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:30.689467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa2460-d0ff-4e32-9225-8ca0b6e5e6fd None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:30.689968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa2460-d0ff-4e32-9225-8ca0b6e5e6fd None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:30.690065 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa2460-d0ff-4e32-9225-8ca0b6e5e6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:30.708407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a9aa2460-d0ff-4e32-9225-8ca0b6e5e6fd None None] MFA Rules not processed for user `fffbf852ddc9455f8a68f80ad118e4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:30.709651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a9aa2460-d0ff-4e32-9225-8ca0b6e5e6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:30.767707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1140/2284] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:11:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:40.761043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=25 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:44.389601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1d745-8eac-4adb-8670-ff6634e0fd53 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:44.390117 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1d745-8eac-4adb-8670-ff6634e0fd53 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:44.390299 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1d745-8eac-4adb-8670-ff6634e0fd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:44.408744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-67d1d745-8eac-4adb-8670-ff6634e0fd53 None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:44.410493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-67d1d745-8eac-4adb-8670-ff6634e0fd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:44.473272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1145/2285] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:11:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:44.528350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa2460-d0ff-4e32-9225-8ca0b6e5e6fd None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:44.529764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a9aa2460-d0ff-4e32-9225-8ca0b6e5e6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:44.613036 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa2460-d0ff-4e32-9225-8ca0b6e5e6fd None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:44.619627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774ca34-e24a-48ac-8b0d-7eada285aa9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:44.620311 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774ca34-e24a-48ac-8b0d-7eada285aa9f service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:44.620595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774ca34-e24a-48ac-8b0d-7eada285aa9f service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:44.620873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774ca34-e24a-48ac-8b0d-7eada285aa9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:44.630085 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774ca34-e24a-48ac-8b0d-7eada285aa9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:44.631724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774ca34-e24a-48ac-8b0d-7eada285aa9f service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:44.640403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1141/2286] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:11:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:11:54.465422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=25 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:54.637378 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=32 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:59.213559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c1199-a7ec-4be1-b93f-15ef8011e461 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:59.214098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c1199-a7ec-4be1-b93f-15ef8011e461 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:59.214098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c1199-a7ec-4be1-b93f-15ef8011e461 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:59.215779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1146/2287] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 11:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:11:59.224249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee6922-d67c-4bd8-9208-af53aef3fc81 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:59.224733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee6922-d67c-4bd8-9208-af53aef3fc81 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:59.225022 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee6922-d67c-4bd8-9208-af53aef3fc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:59.250375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8fee6922-d67c-4bd8-9208-af53aef3fc81 None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:59.251692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8fee6922-d67c-4bd8-9208-af53aef3fc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:59.334150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1142/2288] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 11:11:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:59.352264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c1199-a7ec-4be1-b93f-15ef8011e461 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:59.355024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f15c1199-a7ec-4be1-b93f-15ef8011e461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:59.421661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c1199-a7ec-4be1-b93f-15ef8011e461 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:59.425322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c9ac9-eed9-401d-8d4c-5f1ca4bf9e48 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1ad1ffe4d99420cb568d6154f779838', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:59.425978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c9ac9-eed9-401d-8d4c-5f1ca4bf9e48 service glance] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:59.426263 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c9ac9-eed9-401d-8d4c-5f1ca4bf9e48 service glance] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:59.426546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c9ac9-eed9-401d-8d4c-5f1ca4bf9e48 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:59.437172 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c9ac9-eed9-401d-8d4c-5f1ca4bf9e48 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:59.438610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c9ac9-eed9-401d-8d4c-5f1ca4bf9e48 service glance] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:59.450116 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1147/2289] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:11:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:11:59.523519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cff18-e441-4be3-9337-677f4c1f6cd4 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:59.523928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cff18-e441-4be3-9337-677f4c1f6cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:59.524251 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cff18-e441-4be3-9337-677f4c1f6cd4 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:59.525614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1143/2290] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 11:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:11:59.530468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103e9b9-6591-4d94-aaec-179b63a606d2 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:59.530769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103e9b9-6591-4d94-aaec-179b63a606d2 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:59.531036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103e9b9-6591-4d94-aaec-179b63a606d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:59.550355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3103e9b9-6591-4d94-aaec-179b63a606d2 None None] MFA Rules not processed for user `dadd485b7cac42f4aa079cab91973c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:59.551585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3103e9b9-6591-4d94-aaec-179b63a606d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:59.604034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1148/2291] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 11:11:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:59.617697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2cff18-e441-4be3-9337-677f4c1f6cd4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:59.620083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0d2cff18-e441-4be3-9337-677f4c1f6cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:59.633913 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2cff18-e441-4be3-9337-677f4c1f6cd4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:59.637305 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c774e32-ce7e-41f9-b93d-19b4b4789343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:59.637948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c774e32-ce7e-41f9-b93d-19b4b4789343 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:59.638228 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c774e32-ce7e-41f9-b93d-19b4b4789343 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:59.638502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c774e32-ce7e-41f9-b93d-19b4b4789343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:59.646711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c774e32-ce7e-41f9-b93d-19b4b4789343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:59.648177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c774e32-ce7e-41f9-b93d-19b4b4789343 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:59.656060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1144/2292] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:11:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:11:59.739822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d50945-cd9e-4dbe-a16d-1804893fd6b0 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:59.740109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d50945-cd9e-4dbe-a16d-1804893fd6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:59.740397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d50945-cd9e-4dbe-a16d-1804893fd6b0 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:59.741843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1149/2293] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 11:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:11:59.747809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa9b03-3535-4bf3-bffe-9c1fa556a3e2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:59.748102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa9b03-3535-4bf3-bffe-9c1fa556a3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:59.748382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa9b03-3535-4bf3-bffe-9c1fa556a3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:59.767723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bdfa9b03-3535-4bf3-bffe-9c1fa556a3e2 None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:11:59.769450 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bdfa9b03-3535-4bf3-bffe-9c1fa556a3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:59.779075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1145/2294] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 11:11:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:11:59.791365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d50945-cd9e-4dbe-a16d-1804893fd6b0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:59.803117 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d50945-cd9e-4dbe-a16d-1804893fd6b0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:59.807271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ea9a9-5e23-41ba-a749-b6be4520a340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:59.807968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ea9a9-5e23-41ba-a749-b6be4520a340 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:59.808259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ea9a9-5e23-41ba-a749-b6be4520a340 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:59.808546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ea9a9-5e23-41ba-a749-b6be4520a340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:59.819998 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ea9a9-5e23-41ba-a749-b6be4520a340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:59.821469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ea9a9-5e23-41ba-a749-b6be4520a340 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:59.834097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1150/2295] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:11:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:12:08.687515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2ab20-b6b4-4666-92ee-7b8ce82755cb None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:12:08.688091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2ab20-b6b4-4666-92ee-7b8ce82755cb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:12:08.688183 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2ab20-b6b4-4666-92ee-7b8ce82755cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:12:08.709646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ece2ab20-b6b4-4666-92ee-7b8ce82755cb None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:12:08.710952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ece2ab20-b6b4-4666-92ee-7b8ce82755cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:12:08.718608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1146/2296] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:12:08.733587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ea9a9-5e23-41ba-a749-b6be4520a340 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:12:08.745628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ea9a9-5e23-41ba-a749-b6be4520a340 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:12:08.750757 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6a0ec-e216-46a4-95c2-3a89851357fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:12:08.751443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6a0ec-e216-46a4-95c2-3a89851357fa service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:12:08.751738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6a0ec-e216-46a4-95c2-3a89851357fa service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:12:08.752026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6a0ec-e216-46a4-95c2-3a89851357fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:12:08.761473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6a0ec-e216-46a4-95c2-3a89851357fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:12:08.763075 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6a0ec-e216-46a4-95c2-3a89851357fa service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:12:08.771811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1151/2297] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:12:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:12:11.077725 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2ab20-b6b4-4666-92ee-7b8ce82755cb None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:12:11.079279 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ece2ab20-b6b4-4666-92ee-7b8ce82755cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:12:11.091600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2ab20-b6b4-4666-92ee-7b8ce82755cb None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:12:11.095286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4ab27-1b23-4c6a-9f29-924a0f2cb8c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:12:11.095972 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4ab27-1b23-4c6a-9f29-924a0f2cb8c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:12:11.096310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4ab27-1b23-4c6a-9f29-924a0f2cb8c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:12:11.097105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4ab27-1b23-4c6a-9f29-924a0f2cb8c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:12:11.098902 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3fd4ab27-1b23-4c6a-9f29-924a0f2cb8c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:12:11.112923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4ab27-1b23-4c6a-9f29-924a0f2cb8c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:12:11.114376 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4ab27-1b23-4c6a-9f29-924a0f2cb8c6 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:12:11.127050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1147/2298] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:12:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:12:13.633718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6a0ec-e216-46a4-95c2-3a89851357fa service swift] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:12:13.636264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-06c6a0ec-e216-46a4-95c2-3a89851357fa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:12:13.709952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6a0ec-e216-46a4-95c2-3a89851357fa service swift] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:12:13.713743 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f3aa4-281d-4a27-b188-811f1406cf64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:12:13.714427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f3aa4-281d-4a27-b188-811f1406cf64 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:12:13.714717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f3aa4-281d-4a27-b188-811f1406cf64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:12:13.715207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f3aa4-281d-4a27-b188-811f1406cf64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:12:13.717218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3f7f3aa4-281d-4a27-b188-811f1406cf64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:12:13.733711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f3aa4-281d-4a27-b188-811f1406cf64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:12:13.735155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f3aa4-281d-4a27-b188-811f1406cf64 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:12:13.746937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1152/2299] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:12:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:12:21.121089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=49 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:12:23.741716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=87 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:12:30.112214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4ab27-1b23-4c6a-9f29-924a0f2cb8c6 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:12:30.113721 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3fd4ab27-1b23-4c6a-9f29-924a0f2cb8c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:12:30.127926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4ab27-1b23-4c6a-9f29-924a0f2cb8c6 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:12:30.133328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec9648-2338-41f0-8077-a472390170a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:12:30.133970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec9648-2338-41f0-8077-a472390170a2 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:12:30.134257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec9648-2338-41f0-8077-a472390170a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:12:30.134534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec9648-2338-41f0-8077-a472390170a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:12:30.136206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0dec9648-2338-41f0-8077-a472390170a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:12:30.194628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec9648-2338-41f0-8077-a472390170a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:12:30.196301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec9648-2338-41f0-8077-a472390170a2 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:12:30.206576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1148/2300] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:12:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:12:33.065440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f3aa4-281d-4a27-b188-811f1406cf64 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:12:33.079454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f3aa4-281d-4a27-b188-811f1406cf64 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:12:33.085058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e444c-378c-4461-acf5-2d8d457e320c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:12:33.085709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e444c-378c-4461-acf5-2d8d457e320c service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:12:33.085995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e444c-378c-4461-acf5-2d8d457e320c service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:12:33.086275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e444c-378c-4461-acf5-2d8d457e320c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:12:33.088071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-016e444c-378c-4461-acf5-2d8d457e320c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:12:33.150019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e444c-378c-4461-acf5-2d8d457e320c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:12:33.151672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e444c-378c-4461-acf5-2d8d457e320c service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:12:33.161889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1153/2301] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:12:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:12:37.884321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac53c2a-4458-4d7f-90ff-c01f45f9f16d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:12:37.884859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac53c2a-4458-4d7f-90ff-c01f45f9f16d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:12:37.884859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac53c2a-4458-4d7f-90ff-c01f45f9f16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:12:37.909186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5ac53c2a-4458-4d7f-90ff-c01f45f9f16d None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:12:37.910542 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5ac53c2a-4458-4d7f-90ff-c01f45f9f16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:12:37.917971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1149/2302] 10.222.0.75 () {58 vars in 967 bytes} [Thu Jul 14 11:12:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:12:37.930558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e444c-378c-4461-acf5-2d8d457e320c service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:12:37.941786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e444c-378c-4461-acf5-2d8d457e320c service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:12:37.946977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5a892-2307-4739-b824-d2aae4fbb505 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:12:37.947689 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5a892-2307-4739-b824-d2aae4fbb505 service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:12:37.947972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5a892-2307-4739-b824-d2aae4fbb505 service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:12:37.948253 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5a892-2307-4739-b824-d2aae4fbb505 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:12:37.957394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5a892-2307-4739-b824-d2aae4fbb505 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:12:37.958988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5a892-2307-4739-b824-d2aae4fbb505 service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:12:37.967619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1154/2303] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:12:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:12:47.902569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=31 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:12:47.964094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=34 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:12:48.523682 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916fd7b-8185-4a18-9e26-93fb31e19f71 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:12:48.523973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916fd7b-8185-4a18-9e26-93fb31e19f71 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:12:48.524252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916fd7b-8185-4a18-9e26-93fb31e19f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:12:48.542117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e916fd7b-8185-4a18-9e26-93fb31e19f71 None None] MFA Rules not processed for user `cb80e0c64c3348cfac5ad2c4f82fb606`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:12:48.543326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e916fd7b-8185-4a18-9e26-93fb31e19f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:12:48.550789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1150/2304] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:12:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:12:53.840510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5a892-2307-4739-b824-d2aae4fbb505 service swift] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:12:53.850679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5a892-2307-4739-b824-d2aae4fbb505 service swift] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:12:53.855480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298e891-db85-4d04-81c2-6f69b00aa2e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:12:53.856169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298e891-db85-4d04-81c2-6f69b00aa2e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:12:53.856460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298e891-db85-4d04-81c2-6f69b00aa2e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:12:53.856744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298e891-db85-4d04-81c2-6f69b00aa2e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:12:53.858785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3298e891-db85-4d04-81c2-6f69b00aa2e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:12:53.869699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298e891-db85-4d04-81c2-6f69b00aa2e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:12:53.871142 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298e891-db85-4d04-81c2-6f69b00aa2e4 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:12:53.881126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1155/2305] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:12:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:12:58.536774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:03.875859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=6 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:16.123247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916fd7b-8185-4a18-9e26-93fb31e19f71 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:16.137702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916fd7b-8185-4a18-9e26-93fb31e19f71 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:16.143169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560d3f5-9d50-424a-82ad-9dce1bd7be93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:16.143848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560d3f5-9d50-424a-82ad-9dce1bd7be93 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:16.144140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560d3f5-9d50-424a-82ad-9dce1bd7be93 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:16.144424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560d3f5-9d50-424a-82ad-9dce1bd7be93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:16.146321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2560d3f5-9d50-424a-82ad-9dce1bd7be93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:16.157315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560d3f5-9d50-424a-82ad-9dce1bd7be93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:16.158923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560d3f5-9d50-424a-82ad-9dce1bd7be93 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:16.167826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1151/2306] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:13:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:13:19.755879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298e891-db85-4d04-81c2-6f69b00aa2e4 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:19.776550 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298e891-db85-4d04-81c2-6f69b00aa2e4 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:19.783696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a130a-18ac-43d6-ad40-a65a23f29f7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:19.784376 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a130a-18ac-43d6-ad40-a65a23f29f7a service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:19.784652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a130a-18ac-43d6-ad40-a65a23f29f7a service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:19.784990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a130a-18ac-43d6-ad40-a65a23f29f7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:19.787446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ce1a130a-18ac-43d6-ad40-a65a23f29f7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:19.801468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a130a-18ac-43d6-ad40-a65a23f29f7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:19.803113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a130a-18ac-43d6-ad40-a65a23f29f7a service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:19.826124 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1156/2307] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:13:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:13:26.163708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=6 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:27.983167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f0ec9-3096-43be-9f80-4b52766bb011 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:27.983829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f0ec9-3096-43be-9f80-4b52766bb011 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:27.983829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f0ec9-3096-43be-9f80-4b52766bb011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:28.001794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-579f0ec9-3096-43be-9f80-4b52766bb011 None None] MFA Rules not processed for user `fffbf852ddc9455f8a68f80ad118e4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:13:28.003006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-579f0ec9-3096-43be-9f80-4b52766bb011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:28.010006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1152/2308] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:13:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:13:28.334763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a130a-18ac-43d6-ad40-a65a23f29f7a service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:28.345077 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a130a-18ac-43d6-ad40-a65a23f29f7a service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:28.348800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32994f9d-2afd-4596-b298-71ed0919cbc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:28.349452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32994f9d-2afd-4596-b298-71ed0919cbc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:28.349739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32994f9d-2afd-4596-b298-71ed0919cbc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:28.350021 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32994f9d-2afd-4596-b298-71ed0919cbc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:28.351869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-32994f9d-2afd-4596-b298-71ed0919cbc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:28.362444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32994f9d-2afd-4596-b298-71ed0919cbc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:28.363913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32994f9d-2afd-4596-b298-71ed0919cbc6 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:28.372660 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1157/2309] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:13:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:13:28.688481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f0ec9-3096-43be-9f80-4b52766bb011 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:28.690110 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-579f0ec9-3096-43be-9f80-4b52766bb011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:28.766284 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f0ec9-3096-43be-9f80-4b52766bb011 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:28.771359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e914d-61b2-4cc0-b5c2-66d793888ca7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:28.772053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e914d-61b2-4cc0-b5c2-66d793888ca7 service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:28.772341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e914d-61b2-4cc0-b5c2-66d793888ca7 service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:28.772620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e914d-61b2-4cc0-b5c2-66d793888ca7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:28.774413 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5c8e914d-61b2-4cc0-b5c2-66d793888ca7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:28.786285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e914d-61b2-4cc0-b5c2-66d793888ca7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:28.787911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e914d-61b2-4cc0-b5c2-66d793888ca7 service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:28.798125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1153/2310] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:13:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:13:37.678483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32994f9d-2afd-4596-b298-71ed0919cbc6 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:13:37.694106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32994f9d-2afd-4596-b298-71ed0919cbc6 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:13:37.699917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542bad9-a95e-4e4c-bc2f-d05247b6de04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:13:37.700588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542bad9-a95e-4e4c-bc2f-d05247b6de04 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:37.700872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542bad9-a95e-4e4c-bc2f-d05247b6de04 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:37.701158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542bad9-a95e-4e4c-bc2f-d05247b6de04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:37.703605 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5542bad9-a95e-4e4c-bc2f-d05247b6de04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:37.716448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5542bad9-a95e-4e4c-bc2f-d05247b6de04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:13:37.718085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5542bad9-a95e-4e4c-bc2f-d05247b6de04 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:13:37.729105 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1158/2311] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:13:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:13:38.793645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=35 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:47.724457 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=18 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:13:55.010820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92319f0d-72a9-4e7a-9d18-82e1c1acbab2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:13:55.011435 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92319f0d-72a9-4e7a-9d18-82e1c1acbab2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:13:55.011435 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92319f0d-72a9-4e7a-9d18-82e1c1acbab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:13:55.029787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-92319f0d-72a9-4e7a-9d18-82e1c1acbab2 None None] MFA Rules not processed for user `bf3817e18de149aba43a2f65f37d63f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:13:55.031553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-92319f0d-72a9-4e7a-9d18-82e1c1acbab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:13:55.038673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1154/2312] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:13:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:05.024275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:10.081927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542bad9-a95e-4e4c-bc2f-d05247b6de04 service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:10.098730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542bad9-a95e-4e4c-bc2f-d05247b6de04 service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:10.105453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047c51b-b43f-484e-817e-b161f6c457d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:10.106315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047c51b-b43f-484e-817e-b161f6c457d9 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:10.106626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047c51b-b43f-484e-817e-b161f6c457d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:10.106891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047c51b-b43f-484e-817e-b161f6c457d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:10.110264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1047c51b-b43f-484e-817e-b161f6c457d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:10.123763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047c51b-b43f-484e-817e-b161f6c457d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:10.126056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047c51b-b43f-484e-817e-b161f6c457d9 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:10.137950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1159/2313] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:14:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:14:20.133844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=6 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:21.545034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92319f0d-72a9-4e7a-9d18-82e1c1acbab2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:21.555186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92319f0d-72a9-4e7a-9d18-82e1c1acbab2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:21.558782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4a732-ad22-49bc-a582-4c4838696a90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:21.559471 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4a732-ad22-49bc-a582-4c4838696a90 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:21.559788 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4a732-ad22-49bc-a582-4c4838696a90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:21.560070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4a732-ad22-49bc-a582-4c4838696a90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:21.561851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-09b4a732-ad22-49bc-a582-4c4838696a90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:21.629710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4a732-ad22-49bc-a582-4c4838696a90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:21.631164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4a732-ad22-49bc-a582-4c4838696a90 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:21.640903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1155/2314] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:14:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:14:24.941817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047c51b-b43f-484e-817e-b161f6c457d9 service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:24.944360 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1047c51b-b43f-484e-817e-b161f6c457d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:24.958561 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047c51b-b43f-484e-817e-b161f6c457d9 service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:24.965883 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa2020-8c02-45ff-859b-605b3552a08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:24.966817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa2020-8c02-45ff-859b-605b3552a08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:24.967161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa2020-8c02-45ff-859b-605b3552a08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:24.967627 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa2020-8c02-45ff-859b-605b3552a08f admin admin] PATH_INFO: `/v3/users/cb80e0c64c3348cfac5ad2c4f82fb606` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:24.968979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa2020-8c02-45ff-859b-605b3552a08f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb80e0c64c3348cfac5ad2c4f82fb606)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:24.971779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa2020-8c02-45ff-859b-605b3552a08f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:25.024250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-80fa2020-8c02-45ff-859b-605b3552a08f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb80e0c64c3348cfac5ad2c4f82fb606'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:25.024900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-80fa2020-8c02-45ff-859b-605b3552a08f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb80e0c64c3348cfac5ad2c4f82fb606'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:25.032454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-80fa2020-8c02-45ff-859b-605b3552a08f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb80e0c64c3348cfac5ad2c4f82fb606'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:25.040930 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-80fa2020-8c02-45ff-859b-605b3552a08f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb80e0c64c3348cfac5ad2c4f82fb606'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:25.041551 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-80fa2020-8c02-45ff-859b-605b3552a08f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb80e0c64c3348cfac5ad2c4f82fb606'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:25.059980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1160/2315] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:14:24 2022] DELETE /identity/v3/users/cb80e0c64c3348cfac5ad2c4f82fb606 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:25.238210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4a732-ad22-49bc-a582-4c4838696a90 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:25.249414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4a732-ad22-49bc-a582-4c4838696a90 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:25.308909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5b272-0e3e-4132-bb1f-585b86a4d251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:25.309783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5b272-0e3e-4132-bb1f-585b86a4d251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:25.310137 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5b272-0e3e-4132-bb1f-585b86a4d251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:25.310485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5b272-0e3e-4132-bb1f-585b86a4d251 admin admin] PATH_INFO: `/v3/projects/b3f043134a3d4369983f36769d2ae197` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:25.311722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5b272-0e3e-4132-bb1f-585b86a4d251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f043134a3d4369983f36769d2ae197)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:25.314037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5b272-0e3e-4132-bb1f-585b86a4d251 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:25.361435 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-52c5b272-0e3e-4132-bb1f-585b86a4d251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f043134a3d4369983f36769d2ae197', 'request_id': 'req-52c5b272-0e3e-4132-bb1f-585b86a4d251'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:25.367717 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-52c5b272-0e3e-4132-bb1f-585b86a4d251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f043134a3d4369983f36769d2ae197', 'request_id': 'req-52c5b272-0e3e-4132-bb1f-585b86a4d251'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:25.373174 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-52c5b272-0e3e-4132-bb1f-585b86a4d251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f043134a3d4369983f36769d2ae197', 'request_id': 'req-52c5b272-0e3e-4132-bb1f-585b86a4d251'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:25.385553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-52c5b272-0e3e-4132-bb1f-585b86a4d251 admin admin] The token cache is being invalidate because project b3f043134a3d4369983f36769d2ae197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:14:25.385867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-52c5b272-0e3e-4132-bb1f-585b86a4d251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:25.386487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-52c5b272-0e3e-4132-bb1f-585b86a4d251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:25.388053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1156/2316] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:14:25 2022] DELETE /identity/v3/projects/b3f043134a3d4369983f36769d2ae197 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:25.393473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aaf1b4-b9e9-4a31-871b-35136e0d375f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:25.393837 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aaf1b4-b9e9-4a31-871b-35136e0d375f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:25.394264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aaf1b4-b9e9-4a31-871b-35136e0d375f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:25.412653 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-00aaf1b4-b9e9-4a31-871b-35136e0d375f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:25.413853 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-00aaf1b4-b9e9-4a31-871b-35136e0d375f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:25.474236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1161/2317] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:25.482166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60cb0d-f3b6-4375-9cd0-1cdf727e5311 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:25.482464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60cb0d-f3b6-4375-9cd0-1cdf727e5311 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:25.482758 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60cb0d-f3b6-4375-9cd0-1cdf727e5311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:25.501601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ff60cb0d-f3b6-4375-9cd0-1cdf727e5311 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:25.502818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ff60cb0d-f3b6-4375-9cd0-1cdf727e5311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:25.510966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1157/2318] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:25.515467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aaf1b4-b9e9-4a31-871b-35136e0d375f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:25.525935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aaf1b4-b9e9-4a31-871b-35136e0d375f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:25.530447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788e37b-6509-432d-b9c1-257c3947a0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:25.531098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788e37b-6509-432d-b9c1-257c3947a0ca admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:25.531395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788e37b-6509-432d-b9c1-257c3947a0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:25.531713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788e37b-6509-432d-b9c1-257c3947a0ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:25.532530 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2788e37b-6509-432d-b9c1-257c3947a0ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:14:25.532839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2788e37b-6509-432d-b9c1-257c3947a0ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:25.534447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2788e37b-6509-432d-b9c1-257c3947a0ca admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:25.545838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1162/2319] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:14:25.550258 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60cb0d-f3b6-4375-9cd0-1cdf727e5311 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:25.559846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60cb0d-f3b6-4375-9cd0-1cdf727e5311 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:25.565046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349762d9-371f-42af-9b36-58c5942027dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:25.565643 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349762d9-371f-42af-9b36-58c5942027dc admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:25.565931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349762d9-371f-42af-9b36-58c5942027dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:25.566217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349762d9-371f-42af-9b36-58c5942027dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:25.566910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349762d9-371f-42af-9b36-58c5942027dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:25.568869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349762d9-371f-42af-9b36-58c5942027dc admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:25.583117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:25.598741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1158/2320] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:14:25 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:14:25.602911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788e37b-6509-432d-b9c1-257c3947a0ca admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:25.613142 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788e37b-6509-432d-b9c1-257c3947a0ca admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:25.670641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa85137-ab81-44aa-a1be-8590044d57b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:25.671327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa85137-ab81-44aa-a1be-8590044d57b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:25.671644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa85137-ab81-44aa-a1be-8590044d57b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:25.671919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa85137-ab81-44aa-a1be-8590044d57b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:25.672627 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa85137-ab81-44aa-a1be-8590044d57b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:25.674572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa85137-ab81-44aa-a1be-8590044d57b9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:25.681336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:25.696434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1163/2321] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:14:25 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:14:25.700667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349762d9-371f-42af-9b36-58c5942027dc admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:25.711634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349762d9-371f-42af-9b36-58c5942027dc admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:25.716140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c8bf4-6c91-4e7a-8dee-38cb10768877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:25.716791 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c8bf4-6c91-4e7a-8dee-38cb10768877 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:25.717073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c8bf4-6c91-4e7a-8dee-38cb10768877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:25.717351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c8bf4-6c91-4e7a-8dee-38cb10768877 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:25.718067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c8bf4-6c91-4e7a-8dee-38cb10768877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:25.719719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c8bf4-6c91-4e7a-8dee-38cb10768877 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:25.730998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1159/2322] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:14:25.734974 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa85137-ab81-44aa-a1be-8590044d57b9 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:25.744995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa85137-ab81-44aa-a1be-8590044d57b9 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:25.749894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85556c8a-8285-4c66-aa53-c1baeb83e3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:25.751100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85556c8a-8285-4c66-aa53-c1baeb83e3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:25.751415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85556c8a-8285-4c66-aa53-c1baeb83e3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:25.751689 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85556c8a-8285-4c66-aa53-c1baeb83e3ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:25.752412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85556c8a-8285-4c66-aa53-c1baeb83e3ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:25.754008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85556c8a-8285-4c66-aa53-c1baeb83e3ca admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:25.763956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1164/2323] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:14:25.768613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c8bf4-6c91-4e7a-8dee-38cb10768877 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:25.778796 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c8bf4-6c91-4e7a-8dee-38cb10768877 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:25.783498 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcdf5b-cf22-42c0-bbda-a4c5d5efce97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:25.784379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcdf5b-cf22-42c0-bbda-a4c5d5efce97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:25.784666 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcdf5b-cf22-42c0-bbda-a4c5d5efce97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:25.784953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcdf5b-cf22-42c0-bbda-a4c5d5efce97 admin admin] PATH_INFO: `/v3/projects/4b1efecc7a6c418db13d104c8d37236b/users/8716ec63ee1641839f75c82f3b5d385e/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:25.800088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcdf5b-cf22-42c0-bbda-a4c5d5efce97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b1efecc7a6c418db13d104c8d37236b, user_id=8716ec63ee1641839f75c82f3b5d385e, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:25.802761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcdf5b-cf22-42c0-bbda-a4c5d5efce97 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:25.817289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1160/2324] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:14:25 2022] PUT /identity/v3/projects/4b1efecc7a6c418db13d104c8d37236b/users/8716ec63ee1641839f75c82f3b5d385e/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:25.827172 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f94c78-f421-4296-83ba-d244842e2f3d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:25.827512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f94c78-f421-4296-83ba-d244842e2f3d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:25.827802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f94c78-f421-4296-83ba-d244842e2f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:25.854043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-06f94c78-f421-4296-83ba-d244842e2f3d None None] MFA Rules not processed for user `8716ec63ee1641839f75c82f3b5d385e`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:25.855344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-06f94c78-f421-4296-83ba-d244842e2f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:25.920255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1165/2325] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:25.925263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcdf5b-cf22-42c0-bbda-a4c5d5efce97 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:25.989911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcdf5b-cf22-42c0-bbda-a4c5d5efce97 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:25.994577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492319e-aa32-4cd4-a189-73589ea45741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:25.995234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492319e-aa32-4cd4-a189-73589ea45741 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:25.995625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492319e-aa32-4cd4-a189-73589ea45741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:25.995907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492319e-aa32-4cd4-a189-73589ea45741 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:25.996583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492319e-aa32-4cd4-a189-73589ea45741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:25.998486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492319e-aa32-4cd4-a189-73589ea45741 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:26.023721 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1161/2326] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:14:25 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:14:26.027866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f94c78-f421-4296-83ba-d244842e2f3d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:26.038925 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f94c78-f421-4296-83ba-d244842e2f3d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:26.094575 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e3daf-f296-4b24-aae6-9ed93c3eedd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:26.095198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e3daf-f296-4b24-aae6-9ed93c3eedd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:26.095480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e3daf-f296-4b24-aae6-9ed93c3eedd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:26.095770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e3daf-f296-4b24-aae6-9ed93c3eedd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:26.096466 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e3daf-f296-4b24-aae6-9ed93c3eedd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:26.098387 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e3daf-f296-4b24-aae6-9ed93c3eedd1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:26.118487 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1166/2327] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:14:26 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:14:26.122375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492319e-aa32-4cd4-a189-73589ea45741 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:26.133264 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492319e-aa32-4cd4-a189-73589ea45741 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:26.138040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85340f-9348-4116-8aa4-580d09f76b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:26.138700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85340f-9348-4116-8aa4-580d09f76b6f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:26.138988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85340f-9348-4116-8aa4-580d09f76b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:26.139269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85340f-9348-4116-8aa4-580d09f76b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:26.140024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85340f-9348-4116-8aa4-580d09f76b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:26.141609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85340f-9348-4116-8aa4-580d09f76b6f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:26.152221 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1162/2328] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:14:26.156166 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e3daf-f296-4b24-aae6-9ed93c3eedd1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:26.166745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e3daf-f296-4b24-aae6-9ed93c3eedd1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:26.171346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a67124-e9e1-4d55-9155-0fa0448446c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:26.172254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a67124-e9e1-4d55-9155-0fa0448446c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:26.173163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a67124-e9e1-4d55-9155-0fa0448446c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:26.173434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a67124-e9e1-4d55-9155-0fa0448446c1 admin admin] PATH_INFO: `/v3/projects/203192fc57e44c5c8d7897448489c64c/users/485c358aff1f45acaac0d665e3346df8/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:26.188770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a67124-e9e1-4d55-9155-0fa0448446c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=203192fc57e44c5c8d7897448489c64c, user_id=485c358aff1f45acaac0d665e3346df8, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:26.191973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a67124-e9e1-4d55-9155-0fa0448446c1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:26.209921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1167/2329] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:14:26 2022] PUT /identity/v3/projects/203192fc57e44c5c8d7897448489c64c/users/485c358aff1f45acaac0d665e3346df8/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:26.213849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85340f-9348-4116-8aa4-580d09f76b6f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:26.224246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85340f-9348-4116-8aa4-580d09f76b6f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:26.278722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460faab-6fa0-4cde-b8d1-8135a9060f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:26.279412 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460faab-6fa0-4cde-b8d1-8135a9060f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:26.279685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460faab-6fa0-4cde-b8d1-8135a9060f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:26.279960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460faab-6fa0-4cde-b8d1-8135a9060f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:26.280677 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460faab-6fa0-4cde-b8d1-8135a9060f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:26.282315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460faab-6fa0-4cde-b8d1-8135a9060f92 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:26.292202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1163/2330] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:14:26.296314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a67124-e9e1-4d55-9155-0fa0448446c1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:26.306952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a67124-e9e1-4d55-9155-0fa0448446c1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:26.311592 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac95cf-ed2c-4451-b678-cdca9c90f8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:26.312477 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac95cf-ed2c-4451-b678-cdca9c90f8f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:26.312764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac95cf-ed2c-4451-b678-cdca9c90f8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:26.313584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac95cf-ed2c-4451-b678-cdca9c90f8f9 admin admin] PATH_INFO: `/v3/projects/203192fc57e44c5c8d7897448489c64c/users/485c358aff1f45acaac0d665e3346df8/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:26.315667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac95cf-ed2c-4451-b678-cdca9c90f8f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=203192fc57e44c5c8d7897448489c64c, user_id=485c358aff1f45acaac0d665e3346df8, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:26.318308 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac95cf-ed2c-4451-b678-cdca9c90f8f9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:26.328511 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1168/2331] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:14:26 2022] PUT /identity/v3/projects/203192fc57e44c5c8d7897448489c64c/users/485c358aff1f45acaac0d665e3346df8/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:26.339093 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1717464-2e8e-4152-bfc1-8c89c29cfac9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:26.339403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1717464-2e8e-4152-bfc1-8c89c29cfac9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:26.339718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1717464-2e8e-4152-bfc1-8c89c29cfac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:26.366638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c1717464-2e8e-4152-bfc1-8c89c29cfac9 None None] MFA Rules not processed for user `485c358aff1f45acaac0d665e3346df8`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:26.368421 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c1717464-2e8e-4152-bfc1-8c89c29cfac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:26.385257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:26.436421 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1164/2332] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:26.447814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac95cf-ed2c-4451-b678-cdca9c90f8f9 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:26.449505 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4aac95cf-ed2c-4451-b678-cdca9c90f8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:26.472010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:26.509469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac95cf-ed2c-4451-b678-cdca9c90f8f9 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:26.513258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84baf0-caeb-4400-9c61-39a6e7b97cb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:26.513909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84baf0-caeb-4400-9c61-39a6e7b97cb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:26.514195 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84baf0-caeb-4400-9c61-39a6e7b97cb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:26.514472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84baf0-caeb-4400-9c61-39a6e7b97cb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:26.524014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84baf0-caeb-4400-9c61-39a6e7b97cb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:26.525443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84baf0-caeb-4400-9c61-39a6e7b97cb5 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:26.535343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1169/2333] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:14:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:14:26.606722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1717464-2e8e-4152-bfc1-8c89c29cfac9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:26.616940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1717464-2e8e-4152-bfc1-8c89c29cfac9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:26.620486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d021b-99ad-4b23-98ce-06414f063940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:26.621129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d021b-99ad-4b23-98ce-06414f063940 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:26.621410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d021b-99ad-4b23-98ce-06414f063940 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:26.621767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d021b-99ad-4b23-98ce-06414f063940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:26.681178 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d021b-99ad-4b23-98ce-06414f063940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:26.682662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d021b-99ad-4b23-98ce-06414f063940 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:26.691754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1165/2334] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:14:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:14:36.530749 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=17 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:36.687597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=48 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:37.605005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84baf0-caeb-4400-9c61-39a6e7b97cb5 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:37.675559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84baf0-caeb-4400-9c61-39a6e7b97cb5 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:37.682130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:37.682957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:37.683246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:37.683569 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] PATH_INFO: `/v3/users/8716ec63ee1641839f75c82f3b5d385e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:37.684786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8716ec63ee1641839f75c82f3b5d385e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:37.686959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:37.739319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8716ec63ee1641839f75c82f3b5d385e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:37.739600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8716ec63ee1641839f75c82f3b5d385e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:37.747592 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8716ec63ee1641839f75c82f3b5d385e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:37.757193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8716ec63ee1641839f75c82f3b5d385e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:37.757827 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8716ec63ee1641839f75c82f3b5d385e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:37.778455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1170/2335] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:14:37 2022] DELETE /identity/v3/users/8716ec63ee1641839f75c82f3b5d385e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:37.785195 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbeca6-4dc9-4849-b68d-46f0b8f2c9c5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:37.785813 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbeca6-4dc9-4849-b68d-46f0b8f2c9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:37.785935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbeca6-4dc9-4849-b68d-46f0b8f2c9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:37.804508 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d7bbeca6-4dc9-4849-b68d-46f0b8f2c9c5 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:37.805730 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d7bbeca6-4dc9-4849-b68d-46f0b8f2c9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:37.865710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1166/2336] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:37.875411 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:37.877405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:37.944357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aef0f-9e56-480e-a2c0-72928fc36f7b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:37.948703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbef5f-f5f0-4d7b-8840-73882f39ae47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:37.948879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbef5f-f5f0-4d7b-8840-73882f39ae47 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:37.949175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbef5f-f5f0-4d7b-8840-73882f39ae47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:37.949405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbef5f-f5f0-4d7b-8840-73882f39ae47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:37.959984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbef5f-f5f0-4d7b-8840-73882f39ae47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:37.961436 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbef5f-f5f0-4d7b-8840-73882f39ae47 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:37.972269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1171/2337] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:14:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:14:38.282732 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbeca6-4dc9-4849-b68d-46f0b8f2c9c5 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:38.284895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d7bbeca6-4dc9-4849-b68d-46f0b8f2c9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:38.299469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbeca6-4dc9-4849-b68d-46f0b8f2c9c5 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:38.306000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:38.307005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:38.307358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:38.307742 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] PATH_INFO: `/v3/projects/4b1efecc7a6c418db13d104c8d37236b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:38.309580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b1efecc7a6c418db13d104c8d37236b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:38.311793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:38.366816 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b1efecc7a6c418db13d104c8d37236b', 'request_id': 'req-accc764a-aae9-4d53-b3fe-789d466449d9'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:38.374079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b1efecc7a6c418db13d104c8d37236b', 'request_id': 'req-accc764a-aae9-4d53-b3fe-789d466449d9'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:38.380824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b1efecc7a6c418db13d104c8d37236b', 'request_id': 'req-accc764a-aae9-4d53-b3fe-789d466449d9'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:38.394273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] The token cache is being invalidate because project 4b1efecc7a6c418db13d104c8d37236b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:14:38.394771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:38.395804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:38.397544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1167/2338] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:14:38 2022] DELETE /identity/v3/projects/4b1efecc7a6c418db13d104c8d37236b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:38.403274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbef5f-f5f0-4d7b-8840-73882f39ae47 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:38.415338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbef5f-f5f0-4d7b-8840-73882f39ae47 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:38.437040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:38.486764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a1511-1837-464c-85bf-9bf5d5b9819f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:38.487957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a1511-1837-464c-85bf-9bf5d5b9819f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:38.488482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a1511-1837-464c-85bf-9bf5d5b9819f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:38.488848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a1511-1837-464c-85bf-9bf5d5b9819f admin admin] PATH_INFO: `/v3/users/485c358aff1f45acaac0d665e3346df8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:38.490176 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a1511-1837-464c-85bf-9bf5d5b9819f admin admin] RBAC: Authorizing `identity:delete_user(user_id=485c358aff1f45acaac0d665e3346df8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:38.492399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a1511-1837-464c-85bf-9bf5d5b9819f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:38.554434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-267a1511-1837-464c-85bf-9bf5d5b9819f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '485c358aff1f45acaac0d665e3346df8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:38.555097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-267a1511-1837-464c-85bf-9bf5d5b9819f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '485c358aff1f45acaac0d665e3346df8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:38.562678 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-267a1511-1837-464c-85bf-9bf5d5b9819f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '485c358aff1f45acaac0d665e3346df8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:38.574409 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-267a1511-1837-464c-85bf-9bf5d5b9819f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '485c358aff1f45acaac0d665e3346df8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:38.575009 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-267a1511-1837-464c-85bf-9bf5d5b9819f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '485c358aff1f45acaac0d665e3346df8'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:38.595179 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1172/2339] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:14:38 2022] DELETE /identity/v3/users/485c358aff1f45acaac0d665e3346df8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:38.934771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:38.937781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:38.966805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc764a-aae9-4d53-b3fe-789d466449d9 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:39.054973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e21e08-1048-46fc-9af7-8e391fe21ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:39.056349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21e08-1048-46fc-9af7-8e391fe21ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.057020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21e08-1048-46fc-9af7-8e391fe21ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.058313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21e08-1048-46fc-9af7-8e391fe21ecd admin admin] PATH_INFO: `/v3/projects/203192fc57e44c5c8d7897448489c64c` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.059566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e21e08-1048-46fc-9af7-8e391fe21ecd admin admin] RBAC: Authorizing `identity:delete_project(project_id=203192fc57e44c5c8d7897448489c64c)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:39.062056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e21e08-1048-46fc-9af7-8e391fe21ecd admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:39.108491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:39.128059 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a0e21e08-1048-46fc-9af7-8e391fe21ecd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203192fc57e44c5c8d7897448489c64c', 'request_id': 'req-a0e21e08-1048-46fc-9af7-8e391fe21ecd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:39.134201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a0e21e08-1048-46fc-9af7-8e391fe21ecd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203192fc57e44c5c8d7897448489c64c', 'request_id': 'req-a0e21e08-1048-46fc-9af7-8e391fe21ecd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:39.139773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a0e21e08-1048-46fc-9af7-8e391fe21ecd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203192fc57e44c5c8d7897448489c64c', 'request_id': 'req-a0e21e08-1048-46fc-9af7-8e391fe21ecd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:39.153041 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a0e21e08-1048-46fc-9af7-8e391fe21ecd admin admin] The token cache is being invalidate because project 203192fc57e44c5c8d7897448489c64c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:14:39.153333 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a0e21e08-1048-46fc-9af7-8e391fe21ecd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:39.153964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a0e21e08-1048-46fc-9af7-8e391fe21ecd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:39.155909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1168/2340] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:14:38 2022] DELETE /identity/v3/projects/203192fc57e44c5c8d7897448489c64c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:39.162305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feed74e-8798-44c2-ac8d-ae59b829a7dd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.162305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feed74e-8798-44c2-ac8d-ae59b829a7dd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.162305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feed74e-8798-44c2-ac8d-ae59b829a7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.184064 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2feed74e-8798-44c2-ac8d-ae59b829a7dd None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:39.185422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2feed74e-8798-44c2-ac8d-ae59b829a7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:39.266837 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1173/2341] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:39.274760 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1ff22-81db-4f10-986b-de53a38014f0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.275058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1ff22-81db-4f10-986b-de53a38014f0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.275343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1ff22-81db-4f10-986b-de53a38014f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.294920 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-f9e1ff22-81db-4f10-986b-de53a38014f0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:39.296207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f9e1ff22-81db-4f10-986b-de53a38014f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:39.305402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1169/2342] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:39.309585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feed74e-8798-44c2-ac8d-ae59b829a7dd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:39.320490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feed74e-8798-44c2-ac8d-ae59b829a7dd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:39.325198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0c814-fc28-43cb-b3c1-64d5ca7e09be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:39.325859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0c814-fc28-43cb-b3c1-64d5ca7e09be admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.326147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0c814-fc28-43cb-b3c1-64d5ca7e09be admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.326427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0c814-fc28-43cb-b3c1-64d5ca7e09be admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.327152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0c814-fc28-43cb-b3c1-64d5ca7e09be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:14:39.327492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0c814-fc28-43cb-b3c1-64d5ca7e09be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:39.329099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0c814-fc28-43cb-b3c1-64d5ca7e09be admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:39.341330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1174/2343] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:14:39.346709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1ff22-81db-4f10-986b-de53a38014f0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:39.356522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1ff22-81db-4f10-986b-de53a38014f0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:39.362528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58714b17-c644-4619-b331-816236cc07ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:39.363192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58714b17-c644-4619-b331-816236cc07ff admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.363548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58714b17-c644-4619-b331-816236cc07ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.363793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58714b17-c644-4619-b331-816236cc07ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.364481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58714b17-c644-4619-b331-816236cc07ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:39.367596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58714b17-c644-4619-b331-816236cc07ff admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:39.396623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1170/2344] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:14:39 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:14:39.400580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0c814-fc28-43cb-b3c1-64d5ca7e09be admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:39.412295 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0c814-fc28-43cb-b3c1-64d5ca7e09be admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:39.440286 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:39.475574 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608742c3-d6b4-4f9b-b3c5-750d7dc555eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:39.476238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608742c3-d6b4-4f9b-b3c5-750d7dc555eb admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.476523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608742c3-d6b4-4f9b-b3c5-750d7dc555eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.476801 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608742c3-d6b4-4f9b-b3c5-750d7dc555eb admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.477497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608742c3-d6b4-4f9b-b3c5-750d7dc555eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:39.479425 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608742c3-d6b4-4f9b-b3c5-750d7dc555eb admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:39.500092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1175/2345] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:14:39 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:14:39.504478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58714b17-c644-4619-b331-816236cc07ff admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:39.515650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58714b17-c644-4619-b331-816236cc07ff admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:39.521264 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa938cf2-0c8d-4171-8313-609f8c3374eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:39.522217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa938cf2-0c8d-4171-8313-609f8c3374eb admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.522506 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa938cf2-0c8d-4171-8313-609f8c3374eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.522784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa938cf2-0c8d-4171-8313-609f8c3374eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.523663 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa938cf2-0c8d-4171-8313-609f8c3374eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:39.526246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa938cf2-0c8d-4171-8313-609f8c3374eb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:39.542167 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1171/2346] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:14:39.546164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608742c3-d6b4-4f9b-b3c5-750d7dc555eb admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:39.557438 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608742c3-d6b4-4f9b-b3c5-750d7dc555eb admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:39.562188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59d911-c8e5-4a6f-b052-47630ec75e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:39.562849 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59d911-c8e5-4a6f-b052-47630ec75e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.563231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59d911-c8e5-4a6f-b052-47630ec75e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.563545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59d911-c8e5-4a6f-b052-47630ec75e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.564591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59d911-c8e5-4a6f-b052-47630ec75e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:39.566362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59d911-c8e5-4a6f-b052-47630ec75e50 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:39.577157 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1176/2347] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:14:39.581422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa938cf2-0c8d-4171-8313-609f8c3374eb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:39.591962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa938cf2-0c8d-4171-8313-609f8c3374eb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:39.597115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16930737-c8cf-48bf-90f9-0c70c664e155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:39.597992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16930737-c8cf-48bf-90f9-0c70c664e155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.598308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16930737-c8cf-48bf-90f9-0c70c664e155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.598589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16930737-c8cf-48bf-90f9-0c70c664e155 admin admin] PATH_INFO: `/v3/projects/3319c6fdf2ce4753b459f7438b726966/users/0aa23d115a4d4c8583cf7319023567c2/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.613477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16930737-c8cf-48bf-90f9-0c70c664e155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3319c6fdf2ce4753b459f7438b726966, user_id=0aa23d115a4d4c8583cf7319023567c2, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:39.616178 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16930737-c8cf-48bf-90f9-0c70c664e155 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:39.628646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1172/2348] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:14:39 2022] PUT /identity/v3/projects/3319c6fdf2ce4753b459f7438b726966/users/0aa23d115a4d4c8583cf7319023567c2/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:39.640102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d254113-23ff-43ec-8b11-ee47d0df4e2a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.640607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d254113-23ff-43ec-8b11-ee47d0df4e2a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.640895 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d254113-23ff-43ec-8b11-ee47d0df4e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.677821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-7d254113-23ff-43ec-8b11-ee47d0df4e2a None None] MFA Rules not processed for user `0aa23d115a4d4c8583cf7319023567c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:39.679310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7d254113-23ff-43ec-8b11-ee47d0df4e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:39.760872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1177/2349] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:39.765313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16930737-c8cf-48bf-90f9-0c70c664e155 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:39.825635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16930737-c8cf-48bf-90f9-0c70c664e155 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:39.830646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f85cb-4aff-4b0a-b117-a55b8942506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:39.831298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f85cb-4aff-4b0a-b117-a55b8942506f admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.831609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f85cb-4aff-4b0a-b117-a55b8942506f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.831881 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f85cb-4aff-4b0a-b117-a55b8942506f admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.832589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f85cb-4aff-4b0a-b117-a55b8942506f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:39.834475 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f85cb-4aff-4b0a-b117-a55b8942506f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:39.859341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1173/2350] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:14:39 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:14:39.863332 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d254113-23ff-43ec-8b11-ee47d0df4e2a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:39.874227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d254113-23ff-43ec-8b11-ee47d0df4e2a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:39.930956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc5690-6886-4162-a35e-8ef99faa6655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:39.931655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc5690-6886-4162-a35e-8ef99faa6655 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.931945 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc5690-6886-4162-a35e-8ef99faa6655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.932229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc5690-6886-4162-a35e-8ef99faa6655 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.932923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc5690-6886-4162-a35e-8ef99faa6655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:39.934839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc5690-6886-4162-a35e-8ef99faa6655 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:39.960057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1178/2351] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:14:39 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:14:39.964385 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f85cb-4aff-4b0a-b117-a55b8942506f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:39.975849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f85cb-4aff-4b0a-b117-a55b8942506f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:39.983347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec42e0-19fb-48d9-a511-f40d3064546c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:39.984154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec42e0-19fb-48d9-a511-f40d3064546c admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:39.984637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec42e0-19fb-48d9-a511-f40d3064546c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:39.985894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec42e0-19fb-48d9-a511-f40d3064546c admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:39.985894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec42e0-19fb-48d9-a511-f40d3064546c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:39.987564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec42e0-19fb-48d9-a511-f40d3064546c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:39.999686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1174/2352] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:14:40.004500 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc5690-6886-4162-a35e-8ef99faa6655 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:40.015277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc5690-6886-4162-a35e-8ef99faa6655 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:40.021044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f5a92-dd64-495e-9802-74268a1ccf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:40.023913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f5a92-dd64-495e-9802-74268a1ccf7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:40.023913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f5a92-dd64-495e-9802-74268a1ccf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:40.023913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f5a92-dd64-495e-9802-74268a1ccf7f admin admin] PATH_INFO: `/v3/projects/afa8d6a0950147e7adb9a1e317951a11/users/f6fb0f91c32d4a2b8e0fe00cbd2251ae/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:40.038987 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f5a92-dd64-495e-9802-74268a1ccf7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa8d6a0950147e7adb9a1e317951a11, user_id=f6fb0f91c32d4a2b8e0fe00cbd2251ae, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:40.041742 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f5a92-dd64-495e-9802-74268a1ccf7f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:40.055667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1179/2353] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:14:40 2022] PUT /identity/v3/projects/afa8d6a0950147e7adb9a1e317951a11/users/f6fb0f91c32d4a2b8e0fe00cbd2251ae/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:40.059780 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec42e0-19fb-48d9-a511-f40d3064546c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:40.070524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec42e0-19fb-48d9-a511-f40d3064546c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:40.091931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:40.130002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40506a7a-bfe2-4dc7-9d4d-cef93ca59c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:40.130668 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40506a7a-bfe2-4dc7-9d4d-cef93ca59c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:40.130956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40506a7a-bfe2-4dc7-9d4d-cef93ca59c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:40.131236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40506a7a-bfe2-4dc7-9d4d-cef93ca59c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:40.131995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40506a7a-bfe2-4dc7-9d4d-cef93ca59c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:40.133601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40506a7a-bfe2-4dc7-9d4d-cef93ca59c15 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:40.143954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1175/2354] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:14:40.148095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f5a92-dd64-495e-9802-74268a1ccf7f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:40.153592 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:40.160014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f5a92-dd64-495e-9802-74268a1ccf7f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:40.165106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51614f75-bdc7-4b3b-bd1b-8c6f6feac856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:40.165993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51614f75-bdc7-4b3b-bd1b-8c6f6feac856 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:40.166274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51614f75-bdc7-4b3b-bd1b-8c6f6feac856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:40.166553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51614f75-bdc7-4b3b-bd1b-8c6f6feac856 admin admin] PATH_INFO: `/v3/projects/afa8d6a0950147e7adb9a1e317951a11/users/f6fb0f91c32d4a2b8e0fe00cbd2251ae/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:40.168543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51614f75-bdc7-4b3b-bd1b-8c6f6feac856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa8d6a0950147e7adb9a1e317951a11, user_id=f6fb0f91c32d4a2b8e0fe00cbd2251ae, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:40.171272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51614f75-bdc7-4b3b-bd1b-8c6f6feac856 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:40.183089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1180/2355] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:14:40 2022] PUT /identity/v3/projects/afa8d6a0950147e7adb9a1e317951a11/users/f6fb0f91c32d4a2b8e0fe00cbd2251ae/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:40.194983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46091cf2-2edf-48ba-bc2f-7a5495ef2b99 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:40.195278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46091cf2-2edf-48ba-bc2f-7a5495ef2b99 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:40.195593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46091cf2-2edf-48ba-bc2f-7a5495ef2b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:40.221969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-46091cf2-2edf-48ba-bc2f-7a5495ef2b99 None None] MFA Rules not processed for user `f6fb0f91c32d4a2b8e0fe00cbd2251ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:40.223175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-46091cf2-2edf-48ba-bc2f-7a5495ef2b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:40.289308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1176/2356] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:40 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:40.300522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51614f75-bdc7-4b3b-bd1b-8c6f6feac856 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:40.302216 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-51614f75-bdc7-4b3b-bd1b-8c6f6feac856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:40.359962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51614f75-bdc7-4b3b-bd1b-8c6f6feac856 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:40.363560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e368a-7bb3-4cd6-8a14-4778f606f5ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:40.364208 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e368a-7bb3-4cd6-8a14-4778f606f5ca service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:40.364500 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e368a-7bb3-4cd6-8a14-4778f606f5ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:40.364781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e368a-7bb3-4cd6-8a14-4778f606f5ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:40.374495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e368a-7bb3-4cd6-8a14-4778f606f5ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:40.376489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e368a-7bb3-4cd6-8a14-4778f606f5ca service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:40.385993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1181/2357] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:14:40 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:14:40.767561 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46091cf2-2edf-48ba-bc2f-7a5495ef2b99 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:40.829456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46091cf2-2edf-48ba-bc2f-7a5495ef2b99 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:40.834624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:40.835752 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:40.836044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:40.836323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] PATH_INFO: `/v3/users/0aa23d115a4d4c8583cf7319023567c2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:40.837516 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa23d115a4d4c8583cf7319023567c2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:40.839467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:40.892708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa23d115a4d4c8583cf7319023567c2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:40.894519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:40.904540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa23d115a4d4c8583cf7319023567c2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:40.905339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa23d115a4d4c8583cf7319023567c2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:40.913730 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa23d115a4d4c8583cf7319023567c2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:40.914381 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa23d115a4d4c8583cf7319023567c2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:40.932340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1177/2358] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:14:40 2022] DELETE /identity/v3/users/0aa23d115a4d4c8583cf7319023567c2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:40.936878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ac551-856c-4ba8-aa6e-c6817280b9cb None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:40.937167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ac551-856c-4ba8-aa6e-c6817280b9cb None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:40.937448 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ac551-856c-4ba8-aa6e-c6817280b9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:40.955098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-357ac551-856c-4ba8-aa6e-c6817280b9cb None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:40.956353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-357ac551-856c-4ba8-aa6e-c6817280b9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:41.018105 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1182/2359] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:41.028523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:41.029944 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:41.086706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cdf77-c65b-4dad-ac59-ec4785a1b023 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:41.090364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b444d5-8263-41ba-af9f-1a232de732db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:41.091006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b444d5-8263-41ba-af9f-1a232de732db service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:41.091289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b444d5-8263-41ba-af9f-1a232de732db service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:41.091590 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b444d5-8263-41ba-af9f-1a232de732db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:41.101062 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b444d5-8263-41ba-af9f-1a232de732db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:41.102489 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b444d5-8263-41ba-af9f-1a232de732db service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:41.112469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1178/2360] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:14:41.379817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ac551-856c-4ba8-aa6e-c6817280b9cb None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:41.381349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-357ac551-856c-4ba8-aa6e-c6817280b9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:41.394058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ac551-856c-4ba8-aa6e-c6817280b9cb None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:41.398721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:41.399630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:41.399915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:41.400198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] PATH_INFO: `/v3/projects/3319c6fdf2ce4753b459f7438b726966` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:41.401385 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3319c6fdf2ce4753b459f7438b726966)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:41.403298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:41.450473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3319c6fdf2ce4753b459f7438b726966', 'request_id': 'req-e1ba41ec-354d-4238-a418-a3474918a1c5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:41.457100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3319c6fdf2ce4753b459f7438b726966', 'request_id': 'req-e1ba41ec-354d-4238-a418-a3474918a1c5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:41.462595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3319c6fdf2ce4753b459f7438b726966', 'request_id': 'req-e1ba41ec-354d-4238-a418-a3474918a1c5'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:41.475903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] The token cache is being invalidate because project 3319c6fdf2ce4753b459f7438b726966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:14:41.476196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:41.476834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:41.478667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1183/2361] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:14:41 2022] DELETE /identity/v3/projects/3319c6fdf2ce4753b459f7438b726966 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:41.482718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b444d5-8263-41ba-af9f-1a232de732db service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:41.492439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b444d5-8263-41ba-af9f-1a232de732db service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:41.549816 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c3a1a-ff70-43ef-b117-bb05151453ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:41.550700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c3a1a-ff70-43ef-b117-bb05151453ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:41.550992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c3a1a-ff70-43ef-b117-bb05151453ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:41.551275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c3a1a-ff70-43ef-b117-bb05151453ce admin admin] PATH_INFO: `/v3/users/f6fb0f91c32d4a2b8e0fe00cbd2251ae` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:41.552502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c3a1a-ff70-43ef-b117-bb05151453ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6fb0f91c32d4a2b8e0fe00cbd2251ae)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:41.554689 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c3a1a-ff70-43ef-b117-bb05151453ce admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:41.601615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5d0c3a1a-ff70-43ef-b117-bb05151453ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6fb0f91c32d4a2b8e0fe00cbd2251ae'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:41.609069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5d0c3a1a-ff70-43ef-b117-bb05151453ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6fb0f91c32d4a2b8e0fe00cbd2251ae'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:41.610469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5d0c3a1a-ff70-43ef-b117-bb05151453ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6fb0f91c32d4a2b8e0fe00cbd2251ae'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:41.618315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5d0c3a1a-ff70-43ef-b117-bb05151453ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6fb0f91c32d4a2b8e0fe00cbd2251ae'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:41.619086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5d0c3a1a-ff70-43ef-b117-bb05151453ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6fb0f91c32d4a2b8e0fe00cbd2251ae'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:41.624457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:41.639683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1179/2362] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:14:41 2022] DELETE /identity/v3/users/f6fb0f91c32d4a2b8e0fe00cbd2251ae => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:41.921992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:41.924174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:41.937437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba41ec-354d-4238-a418-a3474918a1c5 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:41.950973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:42.001392 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626ed55-7b8e-44be-b522-ffb8a2708437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:42.002183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626ed55-7b8e-44be-b522-ffb8a2708437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.002470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626ed55-7b8e-44be-b522-ffb8a2708437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.002878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626ed55-7b8e-44be-b522-ffb8a2708437 admin admin] PATH_INFO: `/v3/projects/afa8d6a0950147e7adb9a1e317951a11` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.004095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626ed55-7b8e-44be-b522-ffb8a2708437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa8d6a0950147e7adb9a1e317951a11)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:42.006031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626ed55-7b8e-44be-b522-ffb8a2708437 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:42.052344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8626ed55-7b8e-44be-b522-ffb8a2708437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa8d6a0950147e7adb9a1e317951a11', 'request_id': 'req-8626ed55-7b8e-44be-b522-ffb8a2708437'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:42.058557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8626ed55-7b8e-44be-b522-ffb8a2708437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa8d6a0950147e7adb9a1e317951a11', 'request_id': 'req-8626ed55-7b8e-44be-b522-ffb8a2708437'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:42.064008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8626ed55-7b8e-44be-b522-ffb8a2708437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa8d6a0950147e7adb9a1e317951a11', 'request_id': 'req-8626ed55-7b8e-44be-b522-ffb8a2708437'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:42.076533 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8626ed55-7b8e-44be-b522-ffb8a2708437 admin admin] The token cache is being invalidate because project afa8d6a0950147e7adb9a1e317951a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:14:42.076821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8626ed55-7b8e-44be-b522-ffb8a2708437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:42.077460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-8626ed55-7b8e-44be-b522-ffb8a2708437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:14:42.079672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1184/2363] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:14:41 2022] DELETE /identity/v3/projects/afa8d6a0950147e7adb9a1e317951a11 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:42.084577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dbdff-6b8d-4181-ad20-292e98f62d83 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.085065 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dbdff-6b8d-4181-ad20-292e98f62d83 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.085172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dbdff-6b8d-4181-ad20-292e98f62d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.103825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a73dbdff-6b8d-4181-ad20-292e98f62d83 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:42.105019 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a73dbdff-6b8d-4181-ad20-292e98f62d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:42.162585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1180/2364] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:42.170409 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94345196-1245-472c-b6ea-af281914508b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.170694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94345196-1245-472c-b6ea-af281914508b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.170973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94345196-1245-472c-b6ea-af281914508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.189246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-94345196-1245-472c-b6ea-af281914508b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:42.190447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-94345196-1245-472c-b6ea-af281914508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:42.198474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1185/2365] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:42.202770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dbdff-6b8d-4181-ad20-292e98f62d83 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:42.213399 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dbdff-6b8d-4181-ad20-292e98f62d83 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:42.218049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b93833-f669-454c-9c6a-2554b80b1250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:42.218708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b93833-f669-454c-9c6a-2554b80b1250 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.218994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b93833-f669-454c-9c6a-2554b80b1250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.219859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b93833-f669-454c-9c6a-2554b80b1250 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.220598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b93833-f669-454c-9c6a-2554b80b1250 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:14:42.220909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b93833-f669-454c-9c6a-2554b80b1250 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:42.222514 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b93833-f669-454c-9c6a-2554b80b1250 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:42.234604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1181/2366] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:14:42.238973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94345196-1245-472c-b6ea-af281914508b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:42.248261 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94345196-1245-472c-b6ea-af281914508b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:42.253095 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf926625-c033-483d-ab55-31e2e9952cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:42.253678 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf926625-c033-483d-ab55-31e2e9952cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.253981 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf926625-c033-483d-ab55-31e2e9952cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.254257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf926625-c033-483d-ab55-31e2e9952cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.254947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf926625-c033-483d-ab55-31e2e9952cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:42.256877 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf926625-c033-483d-ab55-31e2e9952cbc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:42.283489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1186/2367] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:14:42 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:14:42.287824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b93833-f669-454c-9c6a-2554b80b1250 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:42.297872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b93833-f669-454c-9c6a-2554b80b1250 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:42.353872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c244da-4d62-400a-b201-e562befbc836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:42.354538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c244da-4d62-400a-b201-e562befbc836 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.354822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c244da-4d62-400a-b201-e562befbc836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.355100 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c244da-4d62-400a-b201-e562befbc836 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.355817 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c244da-4d62-400a-b201-e562befbc836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:42.357731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c244da-4d62-400a-b201-e562befbc836 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:42.378850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1182/2368] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:14:42 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:14:42.383359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf926625-c033-483d-ab55-31e2e9952cbc admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:42.394802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf926625-c033-483d-ab55-31e2e9952cbc admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:42.399466 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3fdb1-9260-4caf-a0d0-b9dbe70d8264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:42.400134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3fdb1-9260-4caf-a0d0-b9dbe70d8264 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.400424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3fdb1-9260-4caf-a0d0-b9dbe70d8264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.400707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3fdb1-9260-4caf-a0d0-b9dbe70d8264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.401437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3fdb1-9260-4caf-a0d0-b9dbe70d8264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:42.403026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3fdb1-9260-4caf-a0d0-b9dbe70d8264 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:42.413510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1187/2369] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:14:42.417452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c244da-4d62-400a-b201-e562befbc836 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:42.427409 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c244da-4d62-400a-b201-e562befbc836 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:42.432184 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252e640-4dde-41b7-8637-16a62dca5167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:42.432831 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252e640-4dde-41b7-8637-16a62dca5167 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.433114 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252e640-4dde-41b7-8637-16a62dca5167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.433391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252e640-4dde-41b7-8637-16a62dca5167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.434104 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252e640-4dde-41b7-8637-16a62dca5167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:42.435729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252e640-4dde-41b7-8637-16a62dca5167 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:42.446131 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1183/2370] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:14:42.450080 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3fdb1-9260-4caf-a0d0-b9dbe70d8264 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:42.460716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3fdb1-9260-4caf-a0d0-b9dbe70d8264 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:42.465461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13525b-e642-4491-b5eb-e9fe75d8b8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:42.466336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13525b-e642-4491-b5eb-e9fe75d8b8cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.466619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13525b-e642-4491-b5eb-e9fe75d8b8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.466896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13525b-e642-4491-b5eb-e9fe75d8b8cf admin admin] PATH_INFO: `/v3/projects/6c80e2858a9a4c399f15e307aabf9869/users/aadd8e5bf3574dba95e3ffcfd3c9c10b/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.481164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13525b-e642-4491-b5eb-e9fe75d8b8cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c80e2858a9a4c399f15e307aabf9869, user_id=aadd8e5bf3574dba95e3ffcfd3c9c10b, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:42.484010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13525b-e642-4491-b5eb-e9fe75d8b8cf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:42.497199 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1188/2371] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:14:42 2022] PUT /identity/v3/projects/6c80e2858a9a4c399f15e307aabf9869/users/aadd8e5bf3574dba95e3ffcfd3c9c10b/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:42.508229 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea11f7-0cd5-442a-83ec-534941a480a6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.508453 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea11f7-0cd5-442a-83ec-534941a480a6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.508735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea11f7-0cd5-442a-83ec-534941a480a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.535636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9fea11f7-0cd5-442a-83ec-534941a480a6 None None] MFA Rules not processed for user `aadd8e5bf3574dba95e3ffcfd3c9c10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:42.536845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9fea11f7-0cd5-442a-83ec-534941a480a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:42.578094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:42.603096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1184/2372] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:42.607796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13525b-e642-4491-b5eb-e9fe75d8b8cf admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:42.653998 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:42.672395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13525b-e642-4491-b5eb-e9fe75d8b8cf admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:42.677255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17836b9b-82db-41ad-a207-a25ea01453f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:42.678194 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17836b9b-82db-41ad-a207-a25ea01453f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.678515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17836b9b-82db-41ad-a207-a25ea01453f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.678822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17836b9b-82db-41ad-a207-a25ea01453f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.679554 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17836b9b-82db-41ad-a207-a25ea01453f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:42.681474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17836b9b-82db-41ad-a207-a25ea01453f9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:42.707434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1189/2373] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:14:42 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:14:42.711769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea11f7-0cd5-442a-83ec-534941a480a6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:42.722477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea11f7-0cd5-442a-83ec-534941a480a6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:42.777087 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca58983-d002-4ef5-b40f-2fc0eb22d5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:42.777681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca58983-d002-4ef5-b40f-2fc0eb22d5f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.777964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca58983-d002-4ef5-b40f-2fc0eb22d5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.778240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca58983-d002-4ef5-b40f-2fc0eb22d5f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.778922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca58983-d002-4ef5-b40f-2fc0eb22d5f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:42.780869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca58983-d002-4ef5-b40f-2fc0eb22d5f0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:42.800645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1185/2374] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:14:42 2022] POST /identity/v3/users => generated 295 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:14:42.804944 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17836b9b-82db-41ad-a207-a25ea01453f9 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:42.815937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17836b9b-82db-41ad-a207-a25ea01453f9 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:42.820540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4101a38-554d-4458-9375-e72104358186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:42.821200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4101a38-554d-4458-9375-e72104358186 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.821492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4101a38-554d-4458-9375-e72104358186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.821777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4101a38-554d-4458-9375-e72104358186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.822504 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4101a38-554d-4458-9375-e72104358186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:42.824177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4101a38-554d-4458-9375-e72104358186 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:42.834673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1190/2375] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:14:42.838864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca58983-d002-4ef5-b40f-2fc0eb22d5f0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:42.849011 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca58983-d002-4ef5-b40f-2fc0eb22d5f0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:42.853662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771f4e6-226a-445c-b86d-873258fcb00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:42.854536 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771f4e6-226a-445c-b86d-873258fcb00e admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.854819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771f4e6-226a-445c-b86d-873258fcb00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.855096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771f4e6-226a-445c-b86d-873258fcb00e admin admin] PATH_INFO: `/v3/projects/e91c73dd93a74d3d9e5966b5d08b510d/users/f19bd79a02da43098e0a8731706e5537/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.870063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771f4e6-226a-445c-b86d-873258fcb00e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91c73dd93a74d3d9e5966b5d08b510d, user_id=f19bd79a02da43098e0a8731706e5537, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:42.872764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771f4e6-226a-445c-b86d-873258fcb00e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:42.897701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1186/2376] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:14:42 2022] PUT /identity/v3/projects/e91c73dd93a74d3d9e5966b5d08b510d/users/f19bd79a02da43098e0a8731706e5537/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:42.901808 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4101a38-554d-4458-9375-e72104358186 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:42.912470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4101a38-554d-4458-9375-e72104358186 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:42.971605 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2144-5190-408f-a3b8-ab5e69a7e26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:42.972275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2144-5190-408f-a3b8-ab5e69a7e26d admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:42.972559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2144-5190-408f-a3b8-ab5e69a7e26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:42.972840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2144-5190-408f-a3b8-ab5e69a7e26d admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:42.973557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b2144-5190-408f-a3b8-ab5e69a7e26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:42.975143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b2144-5190-408f-a3b8-ab5e69a7e26d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:42.985894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1191/2377] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:14:42.990125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771f4e6-226a-445c-b86d-873258fcb00e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:43.001303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771f4e6-226a-445c-b86d-873258fcb00e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:43.005957 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3a512-fb1c-4f5b-bfc5-9b80e7e1ea5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:43.006835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3a512-fb1c-4f5b-bfc5-9b80e7e1ea5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:43.007121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3a512-fb1c-4f5b-bfc5-9b80e7e1ea5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:43.007425 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3a512-fb1c-4f5b-bfc5-9b80e7e1ea5d admin admin] PATH_INFO: `/v3/projects/e91c73dd93a74d3d9e5966b5d08b510d/users/f19bd79a02da43098e0a8731706e5537/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:43.009434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3a512-fb1c-4f5b-bfc5-9b80e7e1ea5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91c73dd93a74d3d9e5966b5d08b510d, user_id=f19bd79a02da43098e0a8731706e5537, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:43.012150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3a512-fb1c-4f5b-bfc5-9b80e7e1ea5d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:43.022486 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1187/2378] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:14:42 2022] PUT /identity/v3/projects/e91c73dd93a74d3d9e5966b5d08b510d/users/f19bd79a02da43098e0a8731706e5537/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:14:43.033641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d0c90-1109-482e-bac5-1bff7b418ff4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:43.033869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d0c90-1109-482e-bac5-1bff7b418ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:43.034151 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d0c90-1109-482e-bac5-1bff7b418ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:43.060273 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-583d0c90-1109-482e-bac5-1bff7b418ff4 None None] MFA Rules not processed for user `f19bd79a02da43098e0a8731706e5537`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:14:43.061473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-583d0c90-1109-482e-bac5-1bff7b418ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:43.129506 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1192/2379] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:14:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:14:43.140784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3a512-fb1c-4f5b-bfc5-9b80e7e1ea5d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:43.142755 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f6e3a512-fb1c-4f5b-bfc5-9b80e7e1ea5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:43.199598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3a512-fb1c-4f5b-bfc5-9b80e7e1ea5d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:43.203256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ac46a-4234-4283-8dd9-008603cb4c8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:43.203952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ac46a-4234-4283-8dd9-008603cb4c8e service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:43.204236 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ac46a-4234-4283-8dd9-008603cb4c8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:43.204513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ac46a-4234-4283-8dd9-008603cb4c8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:43.251301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:43.261661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ac46a-4234-4283-8dd9-008603cb4c8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:43.263098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ac46a-4234-4283-8dd9-008603cb4c8e service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:43.271966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1188/2380] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:14:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:14:48.896970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d0c90-1109-482e-bac5-1bff7b418ff4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:14:48.898360 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-583d0c90-1109-482e-bac5-1bff7b418ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:14:48.911199 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d0c90-1109-482e-bac5-1bff7b418ff4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:14:48.914937 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e5dcd-531f-486e-8c03-fba77aa75373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:14:48.915625 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5dcd-531f-486e-8c03-fba77aa75373 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:14:48.915918 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5dcd-531f-486e-8c03-fba77aa75373 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:14:48.916202 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5dcd-531f-486e-8c03-fba77aa75373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:14:48.925445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e5dcd-531f-486e-8c03-fba77aa75373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:14:48.926882 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e5dcd-531f-486e-8c03-fba77aa75373 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:14:48.936721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1193/2381] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:14:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:14:53.267964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:14:58.932419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:17.961644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ac46a-4234-4283-8dd9-008603cb4c8e service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:18.065336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ac46a-4234-4283-8dd9-008603cb4c8e service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:18.069882 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:18.070747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:18.071031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:18.071303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] PATH_INFO: `/v3/users/aadd8e5bf3574dba95e3ffcfd3c9c10b` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:18.072521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aadd8e5bf3574dba95e3ffcfd3c9c10b)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:18.074458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:18.120419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aadd8e5bf3574dba95e3ffcfd3c9c10b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.127484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aadd8e5bf3574dba95e3ffcfd3c9c10b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.128150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aadd8e5bf3574dba95e3ffcfd3c9c10b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.135131 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aadd8e5bf3574dba95e3ffcfd3c9c10b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.135779 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aadd8e5bf3574dba95e3ffcfd3c9c10b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.153115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1189/2382] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:15:17 2022] DELETE /identity/v3/users/aadd8e5bf3574dba95e3ffcfd3c9c10b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:18.157582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df36c53-76f3-4a4a-8876-52cff9049da6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:18.158051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df36c53-76f3-4a4a-8876-52cff9049da6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:18.158147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df36c53-76f3-4a4a-8876-52cff9049da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:18.207928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9df36c53-76f3-4a4a-8876-52cff9049da6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:18.209145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9df36c53-76f3-4a4a-8876-52cff9049da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:18.269905 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1194/2383] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:18.279739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:18.281189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:18.325285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:18.359801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58583b7b-3fc0-4727-8d9c-b72b0bbc99c6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:18.363708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a84eea-b46c-4228-be91-f204c8d5ca63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:18.364354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a84eea-b46c-4228-be91-f204c8d5ca63 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:18.364642 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a84eea-b46c-4228-be91-f204c8d5ca63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:18.364923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a84eea-b46c-4228-be91-f204c8d5ca63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:18.373850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a84eea-b46c-4228-be91-f204c8d5ca63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:18.375226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a84eea-b46c-4228-be91-f204c8d5ca63 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:18.385244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1190/2384] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:18.660253 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df36c53-76f3-4a4a-8876-52cff9049da6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:18.661803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9df36c53-76f3-4a4a-8876-52cff9049da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:18.680045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df36c53-76f3-4a4a-8876-52cff9049da6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:18.685630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:18.686505 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:18.686795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:18.687070 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] PATH_INFO: `/v3/projects/6c80e2858a9a4c399f15e307aabf9869` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:18.688496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c80e2858a9a4c399f15e307aabf9869)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:18.690416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:18.738552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c80e2858a9a4c399f15e307aabf9869', 'request_id': 'req-7f90365a-3d7e-4957-85f5-155a58a569b3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.744764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c80e2858a9a4c399f15e307aabf9869', 'request_id': 'req-7f90365a-3d7e-4957-85f5-155a58a569b3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.750281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c80e2858a9a4c399f15e307aabf9869', 'request_id': 'req-7f90365a-3d7e-4957-85f5-155a58a569b3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.763178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] The token cache is being invalidate because project 6c80e2858a9a4c399f15e307aabf9869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:15:18.763511 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.764135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.765712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1195/2385] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:15:18 2022] DELETE /identity/v3/projects/6c80e2858a9a4c399f15e307aabf9869 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:18.769754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a84eea-b46c-4228-be91-f204c8d5ca63 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:18.779754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a84eea-b46c-4228-be91-f204c8d5ca63 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:18.835544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c397dcb-7578-4e8a-8c99-f6b306dfaed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:18.836421 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c397dcb-7578-4e8a-8c99-f6b306dfaed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:18.836710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c397dcb-7578-4e8a-8c99-f6b306dfaed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:18.836990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c397dcb-7578-4e8a-8c99-f6b306dfaed7 admin admin] PATH_INFO: `/v3/users/f19bd79a02da43098e0a8731706e5537` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:18.838163 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c397dcb-7578-4e8a-8c99-f6b306dfaed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f19bd79a02da43098e0a8731706e5537)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:18.840126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c397dcb-7578-4e8a-8c99-f6b306dfaed7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:18.885288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8c397dcb-7578-4e8a-8c99-f6b306dfaed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f19bd79a02da43098e0a8731706e5537'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.892570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8c397dcb-7578-4e8a-8c99-f6b306dfaed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f19bd79a02da43098e0a8731706e5537'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.893212 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8c397dcb-7578-4e8a-8c99-f6b306dfaed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f19bd79a02da43098e0a8731706e5537'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.900316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8c397dcb-7578-4e8a-8c99-f6b306dfaed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f19bd79a02da43098e0a8731706e5537'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.900946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-8c397dcb-7578-4e8a-8c99-f6b306dfaed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f19bd79a02da43098e0a8731706e5537'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:18.917673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1191/2386] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:15:18 2022] DELETE /identity/v3/users/f19bd79a02da43098e0a8731706e5537 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:19.201729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:19.203518 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:19.216974 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90365a-3d7e-4957-85f5-155a58a569b3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:19.268655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:19.275864 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0399e2c-4b2f-4532-afef-7fc976047726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:19.277211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0399e2c-4b2f-4532-afef-7fc976047726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.277683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0399e2c-4b2f-4532-afef-7fc976047726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.277982 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0399e2c-4b2f-4532-afef-7fc976047726 admin admin] PATH_INFO: `/v3/projects/e91c73dd93a74d3d9e5966b5d08b510d` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.279479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0399e2c-4b2f-4532-afef-7fc976047726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e91c73dd93a74d3d9e5966b5d08b510d)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:19.281529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0399e2c-4b2f-4532-afef-7fc976047726 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:19.327700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b0399e2c-4b2f-4532-afef-7fc976047726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e91c73dd93a74d3d9e5966b5d08b510d', 'request_id': 'req-b0399e2c-4b2f-4532-afef-7fc976047726'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:19.333861 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b0399e2c-4b2f-4532-afef-7fc976047726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e91c73dd93a74d3d9e5966b5d08b510d', 'request_id': 'req-b0399e2c-4b2f-4532-afef-7fc976047726'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:19.339523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b0399e2c-4b2f-4532-afef-7fc976047726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e91c73dd93a74d3d9e5966b5d08b510d', 'request_id': 'req-b0399e2c-4b2f-4532-afef-7fc976047726'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:19.352272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b0399e2c-4b2f-4532-afef-7fc976047726 admin admin] The token cache is being invalidate because project e91c73dd93a74d3d9e5966b5d08b510d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:15:19.352560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b0399e2c-4b2f-4532-afef-7fc976047726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:19.353177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-b0399e2c-4b2f-4532-afef-7fc976047726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:19.354692 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1196/2387] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:15:19 2022] DELETE /identity/v3/projects/e91c73dd93a74d3d9e5966b5d08b510d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:19.359606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31565b69-ce58-454a-bfb7-b294fd22a771 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.360063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31565b69-ce58-454a-bfb7-b294fd22a771 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.360181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31565b69-ce58-454a-bfb7-b294fd22a771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.407761 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-31565b69-ce58-454a-bfb7-b294fd22a771 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:19.408979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-31565b69-ce58-454a-bfb7-b294fd22a771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:19.422382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:19.470118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1192/2388] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:19.478468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07778010-ec12-4c02-9c44-a7ec96b3adeb None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.478767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07778010-ec12-4c02-9c44-a7ec96b3adeb None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.479053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07778010-ec12-4c02-9c44-a7ec96b3adeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.497355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-07778010-ec12-4c02-9c44-a7ec96b3adeb None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:19.499205 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-07778010-ec12-4c02-9c44-a7ec96b3adeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:19.506832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1197/2389] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:19.511289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31565b69-ce58-454a-bfb7-b294fd22a771 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:19.521522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31565b69-ce58-454a-bfb7-b294fd22a771 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:19.526288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19821fbf-cf83-47b0-a23a-2e0b40dda626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:19.526951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19821fbf-cf83-47b0-a23a-2e0b40dda626 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.527237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19821fbf-cf83-47b0-a23a-2e0b40dda626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.527559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19821fbf-cf83-47b0-a23a-2e0b40dda626 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.528287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19821fbf-cf83-47b0-a23a-2e0b40dda626 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:15:19.528597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19821fbf-cf83-47b0-a23a-2e0b40dda626 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:19.530201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19821fbf-cf83-47b0-a23a-2e0b40dda626 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:19.542720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1193/2390] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:15:19.547041 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07778010-ec12-4c02-9c44-a7ec96b3adeb None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:19.556749 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07778010-ec12-4c02-9c44-a7ec96b3adeb None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:19.561658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ce700-62c1-4003-b7cf-8108a0bae650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:19.562323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ce700-62c1-4003-b7cf-8108a0bae650 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.562613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ce700-62c1-4003-b7cf-8108a0bae650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.562929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ce700-62c1-4003-b7cf-8108a0bae650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.563658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ce700-62c1-4003-b7cf-8108a0bae650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:19.565571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ce700-62c1-4003-b7cf-8108a0bae650 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:19.593098 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1198/2391] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:15:19 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:15:19.597219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19821fbf-cf83-47b0-a23a-2e0b40dda626 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:19.607875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19821fbf-cf83-47b0-a23a-2e0b40dda626 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:19.668079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4324c2b-3459-4580-a817-d089f6767aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:19.668757 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4324c2b-3459-4580-a817-d089f6767aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.669042 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4324c2b-3459-4580-a817-d089f6767aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.669341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4324c2b-3459-4580-a817-d089f6767aeb admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.670010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4324c2b-3459-4580-a817-d089f6767aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:19.671967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4324c2b-3459-4580-a817-d089f6767aeb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:19.695927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1194/2392] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:15:19 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:15:19.697446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ce700-62c1-4003-b7cf-8108a0bae650 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:19.709459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ce700-62c1-4003-b7cf-8108a0bae650 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:19.714368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d359105-bf55-48a8-98ac-0366175d71fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:19.715051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d359105-bf55-48a8-98ac-0366175d71fd admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.715339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d359105-bf55-48a8-98ac-0366175d71fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.715681 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d359105-bf55-48a8-98ac-0366175d71fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.716447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d359105-bf55-48a8-98ac-0366175d71fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:19.718161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d359105-bf55-48a8-98ac-0366175d71fd admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:19.729738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1199/2393] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:15:19.733758 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4324c2b-3459-4580-a817-d089f6767aeb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:19.744076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4324c2b-3459-4580-a817-d089f6767aeb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:19.748920 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9298f-44a2-4d11-873f-f691c3041b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:19.749586 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9298f-44a2-4d11-873f-f691c3041b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.749899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9298f-44a2-4d11-873f-f691c3041b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.750180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9298f-44a2-4d11-873f-f691c3041b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.750908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9298f-44a2-4d11-873f-f691c3041b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:19.752555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9298f-44a2-4d11-873f-f691c3041b39 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:19.763243 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1195/2394] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:15:19.767592 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d359105-bf55-48a8-98ac-0366175d71fd admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:19.778270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d359105-bf55-48a8-98ac-0366175d71fd admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:19.783173 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da758d01-391f-4b25-8457-a7eb4f68a7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:19.784137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da758d01-391f-4b25-8457-a7eb4f68a7b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.784426 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da758d01-391f-4b25-8457-a7eb4f68a7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.784736 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da758d01-391f-4b25-8457-a7eb4f68a7b4 admin admin] PATH_INFO: `/v3/projects/fa50cf0941644235bd9f5f5c45f6dbf3/users/a3b4278cae814f149c4065914679da0e/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.800918 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da758d01-391f-4b25-8457-a7eb4f68a7b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa50cf0941644235bd9f5f5c45f6dbf3, user_id=a3b4278cae814f149c4065914679da0e, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:19.803917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da758d01-391f-4b25-8457-a7eb4f68a7b4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:19.817909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1200/2395] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:15:19 2022] PUT /identity/v3/projects/fa50cf0941644235bd9f5f5c45f6dbf3/users/a3b4278cae814f149c4065914679da0e/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:19.829468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83880cce-f921-40c7-9e98-08fea51b90cd None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:19.830044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83880cce-f921-40c7-9e98-08fea51b90cd None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:19.830470 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83880cce-f921-40c7-9e98-08fea51b90cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:19.857978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-83880cce-f921-40c7-9e98-08fea51b90cd None None] MFA Rules not processed for user `a3b4278cae814f149c4065914679da0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:19.859333 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-83880cce-f921-40c7-9e98-08fea51b90cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:19.927265 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1196/2396] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:19.931997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da758d01-391f-4b25-8457-a7eb4f68a7b4 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:19.997976 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da758d01-391f-4b25-8457-a7eb4f68a7b4 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:20.002845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827fdca-03a5-4352-9be5-ca58716dbd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:20.004048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827fdca-03a5-4352-9be5-ca58716dbd9a admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:20.004339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827fdca-03a5-4352-9be5-ca58716dbd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:20.004647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827fdca-03a5-4352-9be5-ca58716dbd9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:20.005356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827fdca-03a5-4352-9be5-ca58716dbd9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:20.007269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827fdca-03a5-4352-9be5-ca58716dbd9a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:20.033806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1201/2397] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:15:19 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:15:20.037759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83880cce-f921-40c7-9e98-08fea51b90cd None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:20.048634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83880cce-f921-40c7-9e98-08fea51b90cd None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:20.060571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:20.108629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37f917-1d85-4459-88f0-26afd692f32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:20.109309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37f917-1d85-4459-88f0-26afd692f32c admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:20.109592 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37f917-1d85-4459-88f0-26afd692f32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:20.109869 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37f917-1d85-4459-88f0-26afd692f32c admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:20.110566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37f917-1d85-4459-88f0-26afd692f32c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:20.112578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37f917-1d85-4459-88f0-26afd692f32c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:20.134581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1197/2398] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:15:20 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:15:20.138533 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827fdca-03a5-4352-9be5-ca58716dbd9a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:20.149042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827fdca-03a5-4352-9be5-ca58716dbd9a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:20.154183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f4459-2425-4f75-9aed-d50face38de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:20.154859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f4459-2425-4f75-9aed-d50face38de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:20.155149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f4459-2425-4f75-9aed-d50face38de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:20.155470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f4459-2425-4f75-9aed-d50face38de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:20.156206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f4459-2425-4f75-9aed-d50face38de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:20.157826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f4459-2425-4f75-9aed-d50face38de7 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:20.160316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:20.170223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1202/2399] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:15:20.174047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37f917-1d85-4459-88f0-26afd692f32c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:20.185984 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37f917-1d85-4459-88f0-26afd692f32c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:20.191735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f2df8-1bf0-4c9a-b03d-60f00dd43579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:20.192853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f2df8-1bf0-4c9a-b03d-60f00dd43579 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:20.193140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f2df8-1bf0-4c9a-b03d-60f00dd43579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:20.193419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f2df8-1bf0-4c9a-b03d-60f00dd43579 admin admin] PATH_INFO: `/v3/projects/82cdafd7d85e48f693d84bdfa72523ee/users/0e56a55a9ba748f5a2ac1b43b3692485/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:20.209280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f2df8-1bf0-4c9a-b03d-60f00dd43579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82cdafd7d85e48f693d84bdfa72523ee, user_id=0e56a55a9ba748f5a2ac1b43b3692485, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:20.211993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f2df8-1bf0-4c9a-b03d-60f00dd43579 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:20.226112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1198/2400] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:15:20 2022] PUT /identity/v3/projects/82cdafd7d85e48f693d84bdfa72523ee/users/0e56a55a9ba748f5a2ac1b43b3692485/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:20.231056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f4459-2425-4f75-9aed-d50face38de7 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:20.241760 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f4459-2425-4f75-9aed-d50face38de7 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:20.296869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07deb3ae-58f5-46c8-a414-20bd45c53ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:20.297465 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deb3ae-58f5-46c8-a414-20bd45c53ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:20.297757 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deb3ae-58f5-46c8-a414-20bd45c53ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:20.298042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deb3ae-58f5-46c8-a414-20bd45c53ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:20.298766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07deb3ae-58f5-46c8-a414-20bd45c53ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:20.300973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07deb3ae-58f5-46c8-a414-20bd45c53ad6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:20.310988 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1203/2401] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:15:20.315075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f2df8-1bf0-4c9a-b03d-60f00dd43579 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:20.326067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f2df8-1bf0-4c9a-b03d-60f00dd43579 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:20.330616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291dba8-c943-4f34-b39a-225d68df008d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:20.331552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291dba8-c943-4f34-b39a-225d68df008d admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:20.331837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291dba8-c943-4f34-b39a-225d68df008d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:20.332115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291dba8-c943-4f34-b39a-225d68df008d admin admin] PATH_INFO: `/v3/projects/82cdafd7d85e48f693d84bdfa72523ee/users/0e56a55a9ba748f5a2ac1b43b3692485/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:20.334073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291dba8-c943-4f34-b39a-225d68df008d admin admin] RBAC: Authorizing `identity:create_grant(project_id=82cdafd7d85e48f693d84bdfa72523ee, user_id=0e56a55a9ba748f5a2ac1b43b3692485, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:20.336741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291dba8-c943-4f34-b39a-225d68df008d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:20.347037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1199/2402] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:15:20 2022] PUT /identity/v3/projects/82cdafd7d85e48f693d84bdfa72523ee/users/0e56a55a9ba748f5a2ac1b43b3692485/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:20.357126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b889a9-d049-41e9-b17e-b9f0045818e9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:20.357424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b889a9-d049-41e9-b17e-b9f0045818e9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:20.357714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b889a9-d049-41e9-b17e-b9f0045818e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:20.384684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-33b889a9-d049-41e9-b17e-b9f0045818e9 None None] MFA Rules not processed for user `0e56a55a9ba748f5a2ac1b43b3692485`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:20.385985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-33b889a9-d049-41e9-b17e-b9f0045818e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:20.454371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1204/2403] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:20.465526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291dba8-c943-4f34-b39a-225d68df008d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:20.467306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a291dba8-c943-4f34-b39a-225d68df008d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:20.524940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291dba8-c943-4f34-b39a-225d68df008d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:20.528549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a01eb7-3a7e-4ad8-b6de-0509cc247c94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:20.529185 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a01eb7-3a7e-4ad8-b6de-0509cc247c94 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:20.529469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a01eb7-3a7e-4ad8-b6de-0509cc247c94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:20.529749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a01eb7-3a7e-4ad8-b6de-0509cc247c94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:20.539063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a01eb7-3a7e-4ad8-b6de-0509cc247c94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:20.540530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a01eb7-3a7e-4ad8-b6de-0509cc247c94 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:20.549976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1200/2404] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:20.614299 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b889a9-d049-41e9-b17e-b9f0045818e9 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:20.677334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b889a9-d049-41e9-b17e-b9f0045818e9 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:20.682400 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:20.683248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:20.683562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:20.683842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] PATH_INFO: `/v3/users/a3b4278cae814f149c4065914679da0e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:20.685031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b4278cae814f149c4065914679da0e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:20.686961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:20.708408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:20.733957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b4278cae814f149c4065914679da0e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:20.734731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b4278cae814f149c4065914679da0e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:20.742233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b4278cae814f149c4065914679da0e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:20.749274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b4278cae814f149c4065914679da0e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:20.749892 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b4278cae814f149c4065914679da0e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:20.766810 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1205/2405] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:15:20 2022] DELETE /identity/v3/users/a3b4278cae814f149c4065914679da0e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:20.771285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2724ee-cea0-43c4-a262-9b3aa9506cf0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:20.771603 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2724ee-cea0-43c4-a262-9b3aa9506cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:20.771879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2724ee-cea0-43c4-a262-9b3aa9506cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:20.789441 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-be2724ee-cea0-43c4-a262-9b3aa9506cf0 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:20.790629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-be2724ee-cea0-43c4-a262-9b3aa9506cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:20.850145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1201/2406] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:20.860188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:20.861855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:20.920834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faecf3c-7224-4fe2-b3f9-8203d9ab2195 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:20.924526 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48024d9-e4d0-481b-9a43-646a7e562a33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:20.925174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48024d9-e4d0-481b-9a43-646a7e562a33 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:20.925463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48024d9-e4d0-481b-9a43-646a7e562a33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:20.925747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48024d9-e4d0-481b-9a43-646a7e562a33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:20.935229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48024d9-e4d0-481b-9a43-646a7e562a33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:20.936694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48024d9-e4d0-481b-9a43-646a7e562a33 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:20.946452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1206/2407] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:21.231319 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2724ee-cea0-43c4-a262-9b3aa9506cf0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:21.233206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-be2724ee-cea0-43c4-a262-9b3aa9506cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:21.247049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2724ee-cea0-43c4-a262-9b3aa9506cf0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:21.252595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:21.253495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:21.254543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:21.255098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] PATH_INFO: `/v3/projects/fa50cf0941644235bd9f5f5c45f6dbf3` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:21.256970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa50cf0941644235bd9f5f5c45f6dbf3)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:21.258888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:21.266053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:21.309941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa50cf0941644235bd9f5f5c45f6dbf3', 'request_id': 'req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.315981 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa50cf0941644235bd9f5f5c45f6dbf3', 'request_id': 'req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.322661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa50cf0941644235bd9f5f5c45f6dbf3', 'request_id': 'req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.335519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] The token cache is being invalidate because project fa50cf0941644235bd9f5f5c45f6dbf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:15:21.335810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.336428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.337975 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1202/2408] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:15:21 2022] DELETE /identity/v3/projects/fa50cf0941644235bd9f5f5c45f6dbf3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:21.341945 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48024d9-e4d0-481b-9a43-646a7e562a33 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:21.351868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48024d9-e4d0-481b-9a43-646a7e562a33 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:21.407934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6c6a4-d24a-4bba-8338-0eaa45bd4081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:21.408811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6c6a4-d24a-4bba-8338-0eaa45bd4081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:21.409092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6c6a4-d24a-4bba-8338-0eaa45bd4081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:21.409369 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6c6a4-d24a-4bba-8338-0eaa45bd4081 admin admin] PATH_INFO: `/v3/users/0e56a55a9ba748f5a2ac1b43b3692485` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:21.410549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6c6a4-d24a-4bba-8338-0eaa45bd4081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e56a55a9ba748f5a2ac1b43b3692485)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:21.412513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6c6a4-d24a-4bba-8338-0eaa45bd4081 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:21.458578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:21.461403 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-23f6c6a4-d24a-4bba-8338-0eaa45bd4081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e56a55a9ba748f5a2ac1b43b3692485'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.462328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-23f6c6a4-d24a-4bba-8338-0eaa45bd4081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e56a55a9ba748f5a2ac1b43b3692485'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.470706 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-23f6c6a4-d24a-4bba-8338-0eaa45bd4081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e56a55a9ba748f5a2ac1b43b3692485'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.477792 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-23f6c6a4-d24a-4bba-8338-0eaa45bd4081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e56a55a9ba748f5a2ac1b43b3692485'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.478383 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-23f6c6a4-d24a-4bba-8338-0eaa45bd4081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e56a55a9ba748f5a2ac1b43b3692485'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.495796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1207/2409] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:15:21 2022] DELETE /identity/v3/users/0e56a55a9ba748f5a2ac1b43b3692485 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:21.776522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:21.778034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:21.791126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b0cc1-d9c1-4b25-aaf1-0ed84061e124 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:21.850361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:21.851263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:21.851578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:21.851857 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd admin admin] PATH_INFO: `/v3/projects/82cdafd7d85e48f693d84bdfa72523ee` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:21.853100 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=82cdafd7d85e48f693d84bdfa72523ee)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:21.855334 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:21.904592 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82cdafd7d85e48f693d84bdfa72523ee', 'request_id': 'req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.911525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82cdafd7d85e48f693d84bdfa72523ee', 'request_id': 'req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.917526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82cdafd7d85e48f693d84bdfa72523ee', 'request_id': 'req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.930560 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd admin admin] The token cache is being invalidate because project 82cdafd7d85e48f693d84bdfa72523ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:15:21.930851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.931504 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-32fbd40b-a95c-45a4-93d3-a8aa41a0bcbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:21.933098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1203/2410] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:15:21 2022] DELETE /identity/v3/projects/82cdafd7d85e48f693d84bdfa72523ee => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:21.938006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67f4aa-2a01-41ff-aaf3-6575c5385698 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:21.938294 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67f4aa-2a01-41ff-aaf3-6575c5385698 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:21.938578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67f4aa-2a01-41ff-aaf3-6575c5385698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:21.957599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2b67f4aa-2a01-41ff-aaf3-6575c5385698 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:21.958820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2b67f4aa-2a01-41ff-aaf3-6575c5385698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:22.021296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1208/2411] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:22.028996 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80e814-2e0a-475e-b38b-a0bdc8f1609b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:22.029308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80e814-2e0a-475e-b38b-a0bdc8f1609b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:22.029612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80e814-2e0a-475e-b38b-a0bdc8f1609b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:22.048248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7e80e814-2e0a-475e-b38b-a0bdc8f1609b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:22.049457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7e80e814-2e0a-475e-b38b-a0bdc8f1609b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:22.058220 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1204/2412] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:15:22.062418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67f4aa-2a01-41ff-aaf3-6575c5385698 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:22.072155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67f4aa-2a01-41ff-aaf3-6575c5385698 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:22.076716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca10ffd-d99c-4890-8f07-f812b23a374f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:22.077364 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca10ffd-d99c-4890-8f07-f812b23a374f admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:22.077647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca10ffd-d99c-4890-8f07-f812b23a374f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:22.077924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca10ffd-d99c-4890-8f07-f812b23a374f admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:22.078670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca10ffd-d99c-4890-8f07-f812b23a374f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:15:22.078976 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca10ffd-d99c-4890-8f07-f812b23a374f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:22.080599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca10ffd-d99c-4890-8f07-f812b23a374f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:22.092132 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1209/2413] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:15:22.096623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80e814-2e0a-475e-b38b-a0bdc8f1609b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:22.106744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80e814-2e0a-475e-b38b-a0bdc8f1609b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:22.111529 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48c2bf-5902-472d-a848-be1a0969150b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:22.112181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48c2bf-5902-472d-a848-be1a0969150b admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:22.112485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48c2bf-5902-472d-a848-be1a0969150b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:22.112758 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48c2bf-5902-472d-a848-be1a0969150b admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:22.114061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48c2bf-5902-472d-a848-be1a0969150b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:22.116051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48c2bf-5902-472d-a848-be1a0969150b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:22.129709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:22.143237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1205/2414] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:15:22 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:15:22.147549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca10ffd-d99c-4890-8f07-f812b23a374f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:22.157594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca10ffd-d99c-4890-8f07-f812b23a374f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:22.215976 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674e986-99d2-4e5e-a3a5-325d6786aacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:22.216654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674e986-99d2-4e5e-a3a5-325d6786aacf admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:22.216934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674e986-99d2-4e5e-a3a5-325d6786aacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:22.217231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674e986-99d2-4e5e-a3a5-325d6786aacf admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:22.217922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674e986-99d2-4e5e-a3a5-325d6786aacf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:22.219928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674e986-99d2-4e5e-a3a5-325d6786aacf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:22.246470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1210/2415] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:15:22 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:15:22.250670 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48c2bf-5902-472d-a848-be1a0969150b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:22.261701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48c2bf-5902-472d-a848-be1a0969150b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:22.266556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8cf11-1f03-4348-b9bc-b58d80fe5ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:22.267224 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8cf11-1f03-4348-b9bc-b58d80fe5ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:22.267541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8cf11-1f03-4348-b9bc-b58d80fe5ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:22.267824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8cf11-1f03-4348-b9bc-b58d80fe5ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:22.268589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8cf11-1f03-4348-b9bc-b58d80fe5ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:22.270774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8cf11-1f03-4348-b9bc-b58d80fe5ee8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:22.281375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1206/2416] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:15:22.285394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674e986-99d2-4e5e-a3a5-325d6786aacf admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:22.295536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674e986-99d2-4e5e-a3a5-325d6786aacf admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:22.305868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc5fcb-5d4c-4412-b063-3749e4b6f092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:22.306558 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc5fcb-5d4c-4412-b063-3749e4b6f092 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:22.306846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc5fcb-5d4c-4412-b063-3749e4b6f092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:22.307134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc5fcb-5d4c-4412-b063-3749e4b6f092 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:22.307912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc5fcb-5d4c-4412-b063-3749e4b6f092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:22.309532 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc5fcb-5d4c-4412-b063-3749e4b6f092 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:22.320580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1211/2417] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:15:22.324846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8cf11-1f03-4348-b9bc-b58d80fe5ee8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:22.336044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8cf11-1f03-4348-b9bc-b58d80fe5ee8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:22.340783 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94017844-3139-4f20-8312-b2285d64730c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:22.341685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94017844-3139-4f20-8312-b2285d64730c admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:22.341967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94017844-3139-4f20-8312-b2285d64730c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:22.342244 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94017844-3139-4f20-8312-b2285d64730c admin admin] PATH_INFO: `/v3/projects/c68d46ce9773414b946d11c9d6c160bf/users/e0e99d9ff56240a888cd5dd931867099/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:22.358885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94017844-3139-4f20-8312-b2285d64730c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68d46ce9773414b946d11c9d6c160bf, user_id=e0e99d9ff56240a888cd5dd931867099, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:22.361515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94017844-3139-4f20-8312-b2285d64730c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:22.374719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1207/2418] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:15:22 2022] PUT /identity/v3/projects/c68d46ce9773414b946d11c9d6c160bf/users/e0e99d9ff56240a888cd5dd931867099/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:22.439059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96604aca-f46d-40c1-a2ee-97bd31ba9af1 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:22.439698 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96604aca-f46d-40c1-a2ee-97bd31ba9af1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:22.439698 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96604aca-f46d-40c1-a2ee-97bd31ba9af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:22.466084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-96604aca-f46d-40c1-a2ee-97bd31ba9af1 None None] MFA Rules not processed for user `e0e99d9ff56240a888cd5dd931867099`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:22.467300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-96604aca-f46d-40c1-a2ee-97bd31ba9af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:22.480017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:22.535602 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1212/2419] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:22.546111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94017844-3139-4f20-8312-b2285d64730c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:22.610543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94017844-3139-4f20-8312-b2285d64730c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:22.615788 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fe7b3-fefa-465b-8d72-650405c287cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:22.616450 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fe7b3-fefa-465b-8d72-650405c287cf admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:22.616737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fe7b3-fefa-465b-8d72-650405c287cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:22.617019 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fe7b3-fefa-465b-8d72-650405c287cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:22.617734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fe7b3-fefa-465b-8d72-650405c287cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:22.619667 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fe7b3-fefa-465b-8d72-650405c287cf admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:22.645042 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1208/2420] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:15:22 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:15:22.649081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96604aca-f46d-40c1-a2ee-97bd31ba9af1 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:22.659794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96604aca-f46d-40c1-a2ee-97bd31ba9af1 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:22.716447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82bf67-a17c-47f2-a618-a9f13632e7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:22.717118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82bf67-a17c-47f2-a618-a9f13632e7dd admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:22.717406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82bf67-a17c-47f2-a618-a9f13632e7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:22.717684 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82bf67-a17c-47f2-a618-a9f13632e7dd admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:22.718373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82bf67-a17c-47f2-a618-a9f13632e7dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:22.720330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82bf67-a17c-47f2-a618-a9f13632e7dd admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:22.740156 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1213/2421] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:15:22 2022] POST /identity/v3/users => generated 301 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:15:22.744170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fe7b3-fefa-465b-8d72-650405c287cf admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:22.754392 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fe7b3-fefa-465b-8d72-650405c287cf admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:22.759036 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711ce92-e62d-4579-8f24-9e6c4bdccade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:22.759729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711ce92-e62d-4579-8f24-9e6c4bdccade admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:22.760022 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711ce92-e62d-4579-8f24-9e6c4bdccade admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:22.760305 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711ce92-e62d-4579-8f24-9e6c4bdccade admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:22.761054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711ce92-e62d-4579-8f24-9e6c4bdccade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:22.762634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711ce92-e62d-4579-8f24-9e6c4bdccade admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:22.772951 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1209/2422] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:15:22.776921 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82bf67-a17c-47f2-a618-a9f13632e7dd admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:22.787799 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82bf67-a17c-47f2-a618-a9f13632e7dd admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:22.792437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc52dd-7687-4c86-9c53-2295f288d497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:22.793316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc52dd-7687-4c86-9c53-2295f288d497 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:22.793607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc52dd-7687-4c86-9c53-2295f288d497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:22.793890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc52dd-7687-4c86-9c53-2295f288d497 admin admin] PATH_INFO: `/v3/projects/3ccd6325fa504cd8b9dd0d9ed9fd2f66/users/7b67a9c4cee04513be277e69637df4a8/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:22.932486 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc52dd-7687-4c86-9c53-2295f288d497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ccd6325fa504cd8b9dd0d9ed9fd2f66, user_id=7b67a9c4cee04513be277e69637df4a8, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:22.935211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc52dd-7687-4c86-9c53-2295f288d497 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:22.948460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1214/2423] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:15:22 2022] PUT /identity/v3/projects/3ccd6325fa504cd8b9dd0d9ed9fd2f66/users/7b67a9c4cee04513be277e69637df4a8/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:22.952464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711ce92-e62d-4579-8f24-9e6c4bdccade admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:22.962525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711ce92-e62d-4579-8f24-9e6c4bdccade admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:23.019749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f7066-0c07-4324-81af-8c9f95f8540f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:23.020451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f7066-0c07-4324-81af-8c9f95f8540f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:23.020738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f7066-0c07-4324-81af-8c9f95f8540f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:23.021019 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f7066-0c07-4324-81af-8c9f95f8540f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:23.021756 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f7066-0c07-4324-81af-8c9f95f8540f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:23.023372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f7066-0c07-4324-81af-8c9f95f8540f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:23.033046 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1210/2424] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:15:23.037776 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc52dd-7687-4c86-9c53-2295f288d497 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:23.048119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc52dd-7687-4c86-9c53-2295f288d497 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:23.053532 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed988633-350e-4cf8-84f3-950d4e8356bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:23.054415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988633-350e-4cf8-84f3-950d4e8356bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:23.054697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988633-350e-4cf8-84f3-950d4e8356bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:23.054978 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988633-350e-4cf8-84f3-950d4e8356bf admin admin] PATH_INFO: `/v3/projects/3ccd6325fa504cd8b9dd0d9ed9fd2f66/users/7b67a9c4cee04513be277e69637df4a8/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:23.056969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed988633-350e-4cf8-84f3-950d4e8356bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ccd6325fa504cd8b9dd0d9ed9fd2f66, user_id=7b67a9c4cee04513be277e69637df4a8, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:23.059649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed988633-350e-4cf8-84f3-950d4e8356bf admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:23.070442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1215/2425] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:15:23 2022] PUT /identity/v3/projects/3ccd6325fa504cd8b9dd0d9ed9fd2f66/users/7b67a9c4cee04513be277e69637df4a8/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:23.080550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af854e27-d07c-4e1b-8cda-bda0016d4c7e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:23.080850 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af854e27-d07c-4e1b-8cda-bda0016d4c7e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:23.081133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af854e27-d07c-4e1b-8cda-bda0016d4c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:23.109130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-af854e27-d07c-4e1b-8cda-bda0016d4c7e None None] MFA Rules not processed for user `7b67a9c4cee04513be277e69637df4a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:23.110362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-af854e27-d07c-4e1b-8cda-bda0016d4c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:23.129000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:23.183279 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1211/2426] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:23.194859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed988633-350e-4cf8-84f3-950d4e8356bf admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:23.197170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ed988633-350e-4cf8-84f3-950d4e8356bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:23.256351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed988633-350e-4cf8-84f3-950d4e8356bf admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:23.259980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928054a7-86d8-4755-9d88-264ea929c0ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:23.260618 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928054a7-86d8-4755-9d88-264ea929c0ec service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:23.260907 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928054a7-86d8-4755-9d88-264ea929c0ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:23.261189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928054a7-86d8-4755-9d88-264ea929c0ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:23.270053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928054a7-86d8-4755-9d88-264ea929c0ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:23.272019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928054a7-86d8-4755-9d88-264ea929c0ec service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:23.281521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1216/2427] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:23.408086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af854e27-d07c-4e1b-8cda-bda0016d4c7e None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:23.470079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af854e27-d07c-4e1b-8cda-bda0016d4c7e None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:23.474527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:23.475407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:23.475709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:23.475985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] PATH_INFO: `/v3/users/e0e99d9ff56240a888cd5dd931867099` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:23.477184 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0e99d9ff56240a888cd5dd931867099)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:23.479129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:23.545028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0e99d9ff56240a888cd5dd931867099'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:23.553093 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0e99d9ff56240a888cd5dd931867099'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:23.553822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0e99d9ff56240a888cd5dd931867099'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:23.561305 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0e99d9ff56240a888cd5dd931867099'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:23.561981 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0e99d9ff56240a888cd5dd931867099'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:23.579401 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1212/2428] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:15:23 2022] DELETE /identity/v3/users/e0e99d9ff56240a888cd5dd931867099 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:23.584612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cb480-302b-48ca-b799-cb46fd8f5418 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:23.585170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cb480-302b-48ca-b799-cb46fd8f5418 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:23.585170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cb480-302b-48ca-b799-cb46fd8f5418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:23.602879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-660cb480-302b-48ca-b799-cb46fd8f5418 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:23.604140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-660cb480-302b-48ca-b799-cb46fd8f5418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:23.609316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:23.668092 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1217/2429] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:23.678173 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:23.679654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:23.720372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:23.738789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381942c-229f-48d7-b223-09a83b06d384 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:23.742494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d8b8e-daec-422e-b559-aa0ce2aac7f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:23.743151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d8b8e-daec-422e-b559-aa0ce2aac7f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:23.743476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d8b8e-daec-422e-b559-aa0ce2aac7f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:23.743759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d8b8e-daec-422e-b559-aa0ce2aac7f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:23.753426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d8b8e-daec-422e-b559-aa0ce2aac7f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:23.755372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d8b8e-daec-422e-b559-aa0ce2aac7f2 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:23.765472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1213/2430] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:15:24.043053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660cb480-302b-48ca-b799-cb46fd8f5418 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:24.044577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-660cb480-302b-48ca-b799-cb46fd8f5418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:24.059263 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660cb480-302b-48ca-b799-cb46fd8f5418 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:24.064089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:24.065625 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:24.065915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:24.066193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] PATH_INFO: `/v3/projects/c68d46ce9773414b946d11c9d6c160bf` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:24.068076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c68d46ce9773414b946d11c9d6c160bf)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:24.070091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:24.117672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c68d46ce9773414b946d11c9d6c160bf', 'request_id': 'req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.124522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c68d46ce9773414b946d11c9d6c160bf', 'request_id': 'req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.130768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c68d46ce9773414b946d11c9d6c160bf', 'request_id': 'req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.143657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] The token cache is being invalidate because project c68d46ce9773414b946d11c9d6c160bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:15:24.143982 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.144673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.146291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1218/2431] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:15:24 2022] DELETE /identity/v3/projects/c68d46ce9773414b946d11c9d6c160bf => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:24.150225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d8b8e-daec-422e-b559-aa0ce2aac7f2 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:24.160206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d8b8e-daec-422e-b559-aa0ce2aac7f2 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:24.218700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:24.219624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:24.219916 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:24.220200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] PATH_INFO: `/v3/users/7b67a9c4cee04513be277e69637df4a8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:24.221420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b67a9c4cee04513be277e69637df4a8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:24.223373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:24.275180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b67a9c4cee04513be277e69637df4a8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.282588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b67a9c4cee04513be277e69637df4a8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.283282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b67a9c4cee04513be277e69637df4a8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.290986 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b67a9c4cee04513be277e69637df4a8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.291684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b67a9c4cee04513be277e69637df4a8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.310383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1214/2432] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:15:24 2022] DELETE /identity/v3/users/7b67a9c4cee04513be277e69637df4a8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:24.600659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:24.602705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:24.616079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940daf8-56d2-4ff2-ae93-e2b7235f1f50 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:24.676177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:24.677071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:24.677372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:24.677634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] PATH_INFO: `/v3/projects/3ccd6325fa504cd8b9dd0d9ed9fd2f66` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:24.679108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ccd6325fa504cd8b9dd0d9ed9fd2f66)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:24.681046 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:24.721791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:24.731786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ccd6325fa504cd8b9dd0d9ed9fd2f66', 'request_id': 'req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.738529 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ccd6325fa504cd8b9dd0d9ed9fd2f66', 'request_id': 'req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.744196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ccd6325fa504cd8b9dd0d9ed9fd2f66', 'request_id': 'req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.757679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] The token cache is being invalidate because project 3ccd6325fa504cd8b9dd0d9ed9fd2f66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:15:24.757970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.758587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:15:24.760148 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1219/2433] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:15:24 2022] DELETE /identity/v3/projects/3ccd6325fa504cd8b9dd0d9ed9fd2f66 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:15:32.351741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:32.353168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:32.409765 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa56ecf-7632-4c3c-82db-adb666072356 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:32.418981 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:32.460921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a4468-46d4-4585-abaa-c2e97cd85886 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:32.461480 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4468-46d4-4585-abaa-c2e97cd85886 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:32.461762 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4468-46d4-4585-abaa-c2e97cd85886 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:32.462037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4468-46d4-4585-abaa-c2e97cd85886 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:32.463871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f37a4468-46d4-4585-abaa-c2e97cd85886 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:32.526797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a4468-46d4-4585-abaa-c2e97cd85886 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:32.528293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a4468-46d4-4585-abaa-c2e97cd85886 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:32.537809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1215/2434] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:34.754664 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:35.345711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:35.347269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:35.450479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c099f6-0bc4-4d58-9d48-5a1379a8593b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:35.456322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5885375-2818-4eac-9989-f1df9d8c0d2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:35.457005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5885375-2818-4eac-9989-f1df9d8c0d2b service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:35.457296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5885375-2818-4eac-9989-f1df9d8c0d2b service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:35.457578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5885375-2818-4eac-9989-f1df9d8c0d2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:35.460122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b5885375-2818-4eac-9989-f1df9d8c0d2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:35.475695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5885375-2818-4eac-9989-f1df9d8c0d2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:35.477352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5885375-2818-4eac-9989-f1df9d8c0d2b service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:35.488982 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1220/2435] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:38.160513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6f6d8-1f78-484d-9f65-614b68de29b2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:38.161056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6f6d8-1f78-484d-9f65-614b68de29b2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:38.161164 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6f6d8-1f78-484d-9f65-614b68de29b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:38.179576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-55a6f6d8-1f78-484d-9f65-614b68de29b2 None None] MFA Rules not processed for user `5a7c1a5f674c4dfbb969ac37c5102408`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:38.180782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-55a6f6d8-1f78-484d-9f65-614b68de29b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:38.187883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1216/2436] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:15:38.507473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5885375-2818-4eac-9989-f1df9d8c0d2b service placement] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:38.508976 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b5885375-2818-4eac-9989-f1df9d8c0d2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:38.586980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5885375-2818-4eac-9989-f1df9d8c0d2b service placement] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:38.590680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ee6d3-d5ef-41ca-8ee3-155987f091c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:38.591324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ee6d3-d5ef-41ca-8ee3-155987f091c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:38.591639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ee6d3-d5ef-41ca-8ee3-155987f091c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:38.591918 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ee6d3-d5ef-41ca-8ee3-155987f091c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:38.593674 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bc1ee6d3-d5ef-41ca-8ee3-155987f091c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:38.603202 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ee6d3-d5ef-41ca-8ee3-155987f091c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:38.604730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ee6d3-d5ef-41ca-8ee3-155987f091c1 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:38.612759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1221/2437] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:48.173662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=45 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:48.609357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=76 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:54.213218 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd31c4b-329d-4814-b6a9-44b23c2f2609 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:54.213736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd31c4b-329d-4814-b6a9-44b23c2f2609 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:54.213841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd31c4b-329d-4814-b6a9-44b23c2f2609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:54.231776 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-acd31c4b-329d-4814-b6a9-44b23c2f2609 None None] MFA Rules not processed for user `fffbf852ddc9455f8a68f80ad118e4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:15:54.232995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-acd31c4b-329d-4814-b6a9-44b23c2f2609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:54.288457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1217/2438] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:04.281178 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=25 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:23.310589 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532f08-0ed7-4d51-b355-5ad2c7c51469 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:23.311085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532f08-0ed7-4d51-b355-5ad2c7c51469 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:23.311178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532f08-0ed7-4d51-b355-5ad2c7c51469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:23.340454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-db532f08-0ed7-4d51-b355-5ad2c7c51469 None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:23.341857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-db532f08-0ed7-4d51-b355-5ad2c7c51469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:23.403661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1222/2439] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:16:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:23.450015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd31c4b-329d-4814-b6a9-44b23c2f2609 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:23.451442 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-acd31c4b-329d-4814-b6a9-44b23c2f2609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:23.529075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd31c4b-329d-4814-b6a9-44b23c2f2609 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:23.534248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ffe76-a046-4a85-af6a-c58adec6eec7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:23.534905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ffe76-a046-4a85-af6a-c58adec6eec7 service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:23.535190 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ffe76-a046-4a85-af6a-c58adec6eec7 service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:23.535507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ffe76-a046-4a85-af6a-c58adec6eec7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:23.544631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ffe76-a046-4a85-af6a-c58adec6eec7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:23.546256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ffe76-a046-4a85-af6a-c58adec6eec7 service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:23.555329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1218/2440] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:16:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:16:33.396716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=32 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:33.551451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=32 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:38.255358 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34420a1d-a0d0-4278-9fcc-107e8e24fa43 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:38.255936 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34420a1d-a0d0-4278-9fcc-107e8e24fa43 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:38.255936 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34420a1d-a0d0-4278-9fcc-107e8e24fa43 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:38.257390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1223/2441] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 11:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:16:38.263883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671553e-245c-453e-aae9-3c85e2ff1b2c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:38.266217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671553e-245c-453e-aae9-3c85e2ff1b2c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:38.266217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671553e-245c-453e-aae9-3c85e2ff1b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:38.313245 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-7671553e-245c-453e-aae9-3c85e2ff1b2c None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:38.315048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7671553e-245c-453e-aae9-3c85e2ff1b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:38.415892 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1219/2442] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 11:16:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:38.427397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34420a1d-a0d0-4278-9fcc-107e8e24fa43 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:38.429321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-34420a1d-a0d0-4278-9fcc-107e8e24fa43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:38.486175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34420a1d-a0d0-4278-9fcc-107e8e24fa43 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:38.490333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e4ccc-8e97-4806-b1cc-8ecefd3eb447 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1ad1ffe4d99420cb568d6154f779838', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:38.491120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e4ccc-8e97-4806-b1cc-8ecefd3eb447 service glance] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:38.491550 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e4ccc-8e97-4806-b1cc-8ecefd3eb447 service glance] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:38.491846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e4ccc-8e97-4806-b1cc-8ecefd3eb447 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:38.502312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e4ccc-8e97-4806-b1cc-8ecefd3eb447 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:38.504499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e4ccc-8e97-4806-b1cc-8ecefd3eb447 service glance] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:38.515559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1224/2443] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:16:38.581469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59231ec-a14e-4d10-be1b-9c18040fec45 None None] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:38.582525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59231ec-a14e-4d10-be1b-9c18040fec45 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:38.582814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59231ec-a14e-4d10-be1b-9c18040fec45 None None] PATH_INFO: `/` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:38.584272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1220/2444] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 11:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:16:38.591517 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbcb14-af35-43d4-b2bd-6aecc96a1b61 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:38.591873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbcb14-af35-43d4-b2bd-6aecc96a1b61 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:38.592124 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbcb14-af35-43d4-b2bd-6aecc96a1b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:38.610142 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-29bbcb14-af35-43d4-b2bd-6aecc96a1b61 None None] MFA Rules not processed for user `dadd485b7cac42f4aa079cab91973c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:38.611328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-29bbcb14-af35-43d4-b2bd-6aecc96a1b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:38.618246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1225/2445] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 11:16:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:38.630479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59231ec-a14e-4d10-be1b-9c18040fec45 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:38.632293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f59231ec-a14e-4d10-be1b-9c18040fec45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:38.646512 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59231ec-a14e-4d10-be1b-9c18040fec45 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:38.650094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c559a-b3bd-4fb6-bc8e-35256f665ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:38.650726 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c559a-b3bd-4fb6-bc8e-35256f665ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:38.651006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c559a-b3bd-4fb6-bc8e-35256f665ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:38.651277 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c559a-b3bd-4fb6-bc8e-35256f665ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:38.658395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c559a-b3bd-4fb6-bc8e-35256f665ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:38.660032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c559a-b3bd-4fb6-bc8e-35256f665ad4 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:38.668008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1221/2446] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:16:38.758184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19513838-a75a-4cec-855b-3231e3b09358 None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:38.758467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19513838-a75a-4cec-855b-3231e3b09358 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:38.758746 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19513838-a75a-4cec-855b-3231e3b09358 None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:38.760174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1226/2447] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 11:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:16:38.766266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b91be6-099d-4496-8722-d533a7d24015 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:38.766556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b91be6-099d-4496-8722-d533a7d24015 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:38.766839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b91be6-099d-4496-8722-d533a7d24015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:38.786273 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-91b91be6-099d-4496-8722-d533a7d24015 None None] MFA Rules not processed for user `7e572bf785284a8693cf386ced004524`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:38.787500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-91b91be6-099d-4496-8722-d533a7d24015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:38.796634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1222/2448] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 11:16:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:38.807647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19513838-a75a-4cec-855b-3231e3b09358 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:38.819309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19513838-a75a-4cec-855b-3231e3b09358 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:38.823002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04a60f-a224-4308-a3c8-a0af69d7f78d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:38.823679 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04a60f-a224-4308-a3c8-a0af69d7f78d service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:38.823966 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04a60f-a224-4308-a3c8-a0af69d7f78d service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:38.824248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04a60f-a224-4308-a3c8-a0af69d7f78d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:38.834609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04a60f-a224-4308-a3c8-a0af69d7f78d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:38.836056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04a60f-a224-4308-a3c8-a0af69d7f78d service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:38.846773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1227/2449] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:16:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:16:40.497126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d875-e0f0-4d87-9d6f-8eacab734bbd None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:40.497630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d875-e0f0-4d87-9d6f-8eacab734bbd None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:40.497736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d875-e0f0-4d87-9d6f-8eacab734bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:40.515991 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5af1d875-e0f0-4d87-9d6f-8eacab734bbd None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:40.517210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5af1d875-e0f0-4d87-9d6f-8eacab734bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:40.524586 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1223/2450] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:16:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:40.536537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04a60f-a224-4308-a3c8-a0af69d7f78d service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:40.547635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04a60f-a224-4308-a3c8-a0af69d7f78d service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:40.553300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a125504-e284-4e67-9625-5c4e6aa71eab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:40.553935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a125504-e284-4e67-9625-5c4e6aa71eab service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:40.554218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a125504-e284-4e67-9625-5c4e6aa71eab service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:40.554498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a125504-e284-4e67-9625-5c4e6aa71eab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:40.562606 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a125504-e284-4e67-9625-5c4e6aa71eab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:40.564216 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a125504-e284-4e67-9625-5c4e6aa71eab service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:40.572966 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1228/2451] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:16:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 11:16:50.510651 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=52 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:50.569739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=41 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:16:59.331467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a30ea2-bcb4-44c4-a399-5edb9356ab50 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:59.332593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a30ea2-bcb4-44c4-a399-5edb9356ab50 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:59.332593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a30ea2-bcb4-44c4-a399-5edb9356ab50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:59.350906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-29a30ea2-bcb4-44c4-a399-5edb9356ab50 None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:16:59.352209 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-29a30ea2-bcb4-44c4-a399-5edb9356ab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:16:59.359551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1224/2452] 10.222.0.75 () {58 vars in 967 bytes} [Thu Jul 14 11:16:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:16:59.372130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a125504-e284-4e67-9625-5c4e6aa71eab service swift] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:16:59.382766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a125504-e284-4e67-9625-5c4e6aa71eab service swift] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:16:59.387812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ab005-0c32-4c6c-abfa-e4984a29cb39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:16:59.388453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ab005-0c32-4c6c-abfa-e4984a29cb39 service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:16:59.388767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ab005-0c32-4c6c-abfa-e4984a29cb39 service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:16:59.389051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ab005-0c32-4c6c-abfa-e4984a29cb39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:16:59.397334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ab005-0c32-4c6c-abfa-e4984a29cb39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:16:59.398909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ab005-0c32-4c6c-abfa-e4984a29cb39 service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:16:59.407208 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1229/2453] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:16:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:17:09.345034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:09.403785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=6 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:28.072314 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a30ea2-bcb4-44c4-a399-5edb9356ab50 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:28.073738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-29a30ea2-bcb4-44c4-a399-5edb9356ab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:28.111159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a30ea2-bcb4-44c4-a399-5edb9356ab50 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:28.115470 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1546c-fdcc-449e-a6d9-53eb6af9a828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:28.116128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1546c-fdcc-449e-a6d9-53eb6af9a828 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:28.116421 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1546c-fdcc-449e-a6d9-53eb6af9a828 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:28.116695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1546c-fdcc-449e-a6d9-53eb6af9a828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:28.118533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e0b1546c-fdcc-449e-a6d9-53eb6af9a828 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:28.133714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1546c-fdcc-449e-a6d9-53eb6af9a828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:28.135259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1546c-fdcc-449e-a6d9-53eb6af9a828 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:28.148219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1225/2454] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:37.513388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ab005-0c32-4c6c-abfa-e4984a29cb39 service swift] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:17:37.524089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ab005-0c32-4c6c-abfa-e4984a29cb39 service swift] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:17:37.528508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6146ef-d5c5-4f5a-81aa-2a7c19cd33b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:17:37.529478 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6146ef-d5c5-4f5a-81aa-2a7c19cd33b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:17:37.529915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6146ef-d5c5-4f5a-81aa-2a7c19cd33b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:17:37.530361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6146ef-d5c5-4f5a-81aa-2a7c19cd33b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:17:37.532630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8d6146ef-d5c5-4f5a-81aa-2a7c19cd33b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:17:37.548297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6146ef-d5c5-4f5a-81aa-2a7c19cd33b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:17:37.550301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6146ef-d5c5-4f5a-81aa-2a7c19cd33b8 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:17:37.562753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1230/2455] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:17:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:17:38.142079 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=15 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:17:47.557483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=6 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:04.910141 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1546c-fdcc-449e-a6d9-53eb6af9a828 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:04.911751 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e0b1546c-fdcc-449e-a6d9-53eb6af9a828 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:04.923487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1546c-fdcc-449e-a6d9-53eb6af9a828 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:04.928040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9341d2-8cad-467d-a2d6-5da4d3a7735f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:04.929113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9341d2-8cad-467d-a2d6-5da4d3a7735f service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:04.929494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9341d2-8cad-467d-a2d6-5da4d3a7735f service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:04.929885 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9341d2-8cad-467d-a2d6-5da4d3a7735f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:04.932032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ba9341d2-8cad-467d-a2d6-5da4d3a7735f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:05.035627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9341d2-8cad-467d-a2d6-5da4d3a7735f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:05.037127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9341d2-8cad-467d-a2d6-5da4d3a7735f service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:05.070541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1226/2456] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:06.632624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6146ef-d5c5-4f5a-81aa-2a7c19cd33b8 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:06.649847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6146ef-d5c5-4f5a-81aa-2a7c19cd33b8 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:06.655858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2265f-8d9a-4603-8c88-56e3c5cbc694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:06.656989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2265f-8d9a-4603-8c88-56e3c5cbc694 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:06.657483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2265f-8d9a-4603-8c88-56e3c5cbc694 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:06.657977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2265f-8d9a-4603-8c88-56e3c5cbc694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:06.661908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a3c2265f-8d9a-4603-8c88-56e3c5cbc694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:06.774956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2265f-8d9a-4603-8c88-56e3c5cbc694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:06.776384 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2265f-8d9a-4603-8c88-56e3c5cbc694 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:06.788771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1231/2457] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:07.310034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9341d2-8cad-467d-a2d6-5da4d3a7735f service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:07.326482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9341d2-8cad-467d-a2d6-5da4d3a7735f service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:07.334539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53d6fb-b90b-4860-8300-55650b80e5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:07.336374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53d6fb-b90b-4860-8300-55650b80e5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:07.337374 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53d6fb-b90b-4860-8300-55650b80e5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:07.338138 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53d6fb-b90b-4860-8300-55650b80e5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:07.342770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ac53d6fb-b90b-4860-8300-55650b80e5d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:07.368527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53d6fb-b90b-4860-8300-55650b80e5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:07.371214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53d6fb-b90b-4860-8300-55650b80e5d4 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:07.391313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1227/2458] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:18:13.467711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371d1e1-02d9-443e-bb72-51bc45da499b None None] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:13.468151 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371d1e1-02d9-443e-bb72-51bc45da499b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:13.468733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371d1e1-02d9-443e-bb72-51bc45da499b None None] PATH_INFO: `/` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:13.476463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1232/2459] 10.222.0.75 () {54 vars in 921 bytes} [Thu Jul 14 11:18:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:18:13.484253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800586b-9660-4b7e-a3ca-bf48a0c6a575 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:13.484914 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800586b-9660-4b7e-a3ca-bf48a0c6a575 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:13.484914 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800586b-9660-4b7e-a3ca-bf48a0c6a575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:13.528860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6800586b-9660-4b7e-a3ca-bf48a0c6a575 None None] MFA Rules not processed for user `0fc1f2abce4845219741f4f48bf32d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:13.530238 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6800586b-9660-4b7e-a3ca-bf48a0c6a575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:13.546684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1228/2460] 10.222.0.75 () {58 vars in 1018 bytes} [Thu Jul 14 11:18:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:13.562868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371d1e1-02d9-443e-bb72-51bc45da499b None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:13.588549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371d1e1-02d9-443e-bb72-51bc45da499b None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:13.594527 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34177b69-2688-4433-9aea-fe250901ccc7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:13.595336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34177b69-2688-4433-9aea-fe250901ccc7 service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:13.595878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34177b69-2688-4433-9aea-fe250901ccc7 service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:13.596831 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34177b69-2688-4433-9aea-fe250901ccc7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:13.608292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34177b69-2688-4433-9aea-fe250901ccc7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:13.610976 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34177b69-2688-4433-9aea-fe250901ccc7 service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:13.620759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800586b-9660-4b7e-a3ca-bf48a0c6a575 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:13.622484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6800586b-9660-4b7e-a3ca-bf48a0c6a575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:13.641124 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1233/2461] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:13.659604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800586b-9660-4b7e-a3ca-bf48a0c6a575 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:13.666569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:13.667694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:13.668207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:13.668598 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] PATH_INFO: `/v3/users/5a7c1a5f674c4dfbb969ac37c5102408` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:13.670129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7c1a5f674c4dfbb969ac37c5102408)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:13.672756 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:13.757252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7c1a5f674c4dfbb969ac37c5102408'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:13.771330 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7c1a5f674c4dfbb969ac37c5102408'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:13.772687 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7c1a5f674c4dfbb969ac37c5102408'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:13.789650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7c1a5f674c4dfbb969ac37c5102408'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:13.790690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7c1a5f674c4dfbb969ac37c5102408'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:13.827044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1229/2462] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:13 2022] DELETE /identity/v3/users/5a7c1a5f674c4dfbb969ac37c5102408 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:14.141681 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34177b69-2688-4433-9aea-fe250901ccc7 service placement] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:14.155368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34177b69-2688-4433-9aea-fe250901ccc7 service placement] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:14.251326 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:14.253207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:14.253754 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:14.254320 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f admin admin] PATH_INFO: `/v3/projects/a7cf89df79d64153a73a5ec4d49f376f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:14.256573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7cf89df79d64153a73a5ec4d49f376f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:14.259317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:14.328249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7cf89df79d64153a73a5ec4d49f376f', 'request_id': 'req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.338368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7cf89df79d64153a73a5ec4d49f376f', 'request_id': 'req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.349511 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7cf89df79d64153a73a5ec4d49f376f', 'request_id': 'req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.359742 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:14.363715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:14.375765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f admin admin] The token cache is being invalidate because project a7cf89df79d64153a73a5ec4d49f376f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:14.376276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.378845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9e022b88-bed3-4bde-9038-4ed2b6b6a24f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.383578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1234/2463] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:14 2022] DELETE /identity/v3/projects/a7cf89df79d64153a73a5ec4d49f376f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:14.388865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328015a-0aa9-4bd8-9e15-6a0d2ba7544a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:14.389168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328015a-0aa9-4bd8-9e15-6a0d2ba7544a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:14.389482 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328015a-0aa9-4bd8-9e15-6a0d2ba7544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:14.392387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1638b4a-d3cb-4229-9cc3-746693b9c2de admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:14.397995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:14.414433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c328015a-0aa9-4bd8-9e15-6a0d2ba7544a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:14.415922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c328015a-0aa9-4bd8-9e15-6a0d2ba7544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:14.425980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:14.477939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:14.478739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:14.479027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:14.479310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] PATH_INFO: `/v3/users/fffbf852ddc9455f8a68f80ad118e4d1` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:14.496021 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fffbf852ddc9455f8a68f80ad118e4d1)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:14.498103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:14.509813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1235/2464] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:14.522733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a3a43-f43c-4026-9be0-9750e875c596 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:14.523025 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a3a43-f43c-4026-9be0-9750e875c596 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:14.523339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a3a43-f43c-4026-9be0-9750e875c596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:14.543460 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3a8a3a43-f43c-4026-9be0-9750e875c596 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:14.544714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3a8a3a43-f43c-4026-9be0-9750e875c596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:14.553899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1236/2465] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:14.558473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a3a43-f43c-4026-9be0-9750e875c596 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:14.562729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fffbf852ddc9455f8a68f80ad118e4d1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.569493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a3a43-f43c-4026-9be0-9750e875c596 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:14.574162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fffbf852ddc9455f8a68f80ad118e4d1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.574859 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fffbf852ddc9455f8a68f80ad118e4d1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.575474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3f781-025d-4782-9c4d-f34f88a5522c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:14.576204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3f781-025d-4782-9c4d-f34f88a5522c admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:14.576499 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3f781-025d-4782-9c4d-f34f88a5522c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:14.576773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3f781-025d-4782-9c4d-f34f88a5522c admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:14.577532 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3f781-025d-4782-9c4d-f34f88a5522c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:14.577844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3f781-025d-4782-9c4d-f34f88a5522c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:14.579594 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3f781-025d-4782-9c4d-f34f88a5522c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:14.584866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fffbf852ddc9455f8a68f80ad118e4d1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.585693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fffbf852ddc9455f8a68f80ad118e4d1'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:14.594662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1237/2466] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:18:14.600547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3f781-025d-4782-9c4d-f34f88a5522c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:14.603161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-24d3f781-025d-4782-9c4d-f34f88a5522c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:14.610286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1230/2467] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:14 2022] DELETE /identity/v3/users/fffbf852ddc9455f8a68f80ad118e4d1 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:14.618841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3f781-025d-4782-9c4d-f34f88a5522c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:14.690718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4bdd9-8177-4263-bad0-e1606853b576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:14.691443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4bdd9-8177-4263-bad0-e1606853b576 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:14.691738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4bdd9-8177-4263-bad0-e1606853b576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:14.692023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4bdd9-8177-4263-bad0-e1606853b576 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:14.692737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4bdd9-8177-4263-bad0-e1606853b576 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:14.694731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4bdd9-8177-4263-bad0-e1606853b576 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:14.746411 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1238/2468] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:14 2022] POST /identity/v3/projects => generated 403 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:14.748595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:14.762804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f74912-616f-468a-8463-cf19ce58647e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:14.869722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55412115-9896-4ffa-9d40-99bcb6696184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:14.870541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55412115-9896-4ffa-9d40-99bcb6696184 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:14.870829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55412115-9896-4ffa-9d40-99bcb6696184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:14.871109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55412115-9896-4ffa-9d40-99bcb6696184 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:14.871865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55412115-9896-4ffa-9d40-99bcb6696184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:14.879605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55412115-9896-4ffa-9d40-99bcb6696184 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:14.884699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4bdd9-8177-4263-bad0-e1606853b576 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:14.892761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e3e4bdd9-8177-4263-bad0-e1606853b576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:14.929422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1231/2469] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:14 2022] POST /identity/v3/users => generated 302 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:14.935728 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55412115-9896-4ffa-9d40-99bcb6696184 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:14.936220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4bdd9-8177-4263-bad0-e1606853b576 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:14.942319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:14.943236 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:14.943556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:14.943836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] PATH_INFO: `/v3/projects/7f04452d94114c64b9882cfa66e29479` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:14.955316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55412115-9896-4ffa-9d40-99bcb6696184 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:14.959521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f04452d94114c64b9882cfa66e29479)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:14.961763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:14.964466 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd6a9d-2f9c-47f8-9b50-4f1065d72435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:14.965604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd6a9d-2f9c-47f8-9b50-4f1065d72435 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:14.966108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd6a9d-2f9c-47f8-9b50-4f1065d72435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:14.967748 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd6a9d-2f9c-47f8-9b50-4f1065d72435 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:14.968725 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd6a9d-2f9c-47f8-9b50-4f1065d72435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:14.970520 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd6a9d-2f9c-47f8-9b50-4f1065d72435 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:14.973903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:14.991464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1232/2470] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:14.995103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd6a9d-2f9c-47f8-9b50-4f1065d72435 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.013533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd6a9d-2f9c-47f8-9b50-4f1065d72435 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.026946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc167b2-7fc3-44c1-b100-0749105c2a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.032170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc167b2-7fc3-44c1-b100-0749105c2a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.032499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc167b2-7fc3-44c1-b100-0749105c2a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.032721 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc167b2-7fc3-44c1-b100-0749105c2a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.033740 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc167b2-7fc3-44c1-b100-0749105c2a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.035958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc167b2-7fc3-44c1-b100-0749105c2a09 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.046141 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f04452d94114c64b9882cfa66e29479', 'request_id': 'req-a9e9688a-622b-4cc8-bb8a-2800acde1adc'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:15.047332 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1233/2471] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:15.048248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc167b2-7fc3-44c1-b100-0749105c2a09 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.050470 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7bc167b2-7fc3-44c1-b100-0749105c2a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:15.058149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f04452d94114c64b9882cfa66e29479', 'request_id': 'req-a9e9688a-622b-4cc8-bb8a-2800acde1adc'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:15.059756 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:15.066759 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f04452d94114c64b9882cfa66e29479', 'request_id': 'req-a9e9688a-622b-4cc8-bb8a-2800acde1adc'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:15.089769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] The token cache is being invalidate because project 7f04452d94114c64b9882cfa66e29479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:15.090263 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:15.091086 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:15.093025 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1239/2472] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:14 2022] DELETE /identity/v3/projects/7f04452d94114c64b9882cfa66e29479 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:15.094042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.095622 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:15.137946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc167b2-7fc3-44c1-b100-0749105c2a09 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.178551 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9688a-622b-4cc8-bb8a-2800acde1adc admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.183890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d9179-0b8e-4ed3-8560-5a32e4eef6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.184846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d9179-0b8e-4ed3-8560-5a32e4eef6e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.185150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d9179-0b8e-4ed3-8560-5a32e4eef6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.185435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d9179-0b8e-4ed3-8560-5a32e4eef6e0 admin admin] PATH_INFO: `/v3/projects/3d52af11ee044ecfb62858553f57d3f3/users/c178bb17bdbc4021bba9114784482c58/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.198350 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ede11-5446-4cf7-bba2-ee1a0356251b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.199349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ede11-5446-4cf7-bba2-ee1a0356251b service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.199830 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ede11-5446-4cf7-bba2-ee1a0356251b service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.200265 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ede11-5446-4cf7-bba2-ee1a0356251b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.202700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-500ede11-5446-4cf7-bba2-ee1a0356251b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:15.213640 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d9179-0b8e-4ed3-8560-5a32e4eef6e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d52af11ee044ecfb62858553f57d3f3, user_id=c178bb17bdbc4021bba9114784482c58, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.216240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ede11-5446-4cf7-bba2-ee1a0356251b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.217621 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d9179-0b8e-4ed3-8560-5a32e4eef6e0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.217771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ede11-5446-4cf7-bba2-ee1a0356251b service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.230454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1234/2473] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:15.232133 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d27e4e-01ae-4d1a-b80e-ccf124311c5b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.232297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d27e4e-01ae-4d1a-b80e-ccf124311c5b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.232581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d27e4e-01ae-4d1a-b80e-ccf124311c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.239440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1240/2474] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:15 2022] PUT /identity/v3/projects/3d52af11ee044ecfb62858553f57d3f3/users/c178bb17bdbc4021bba9114784482c58/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:15.253517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c7d27e4e-01ae-4d1a-b80e-ccf124311c5b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:15.254797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c7d27e4e-01ae-4d1a-b80e-ccf124311c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:15.259452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b199a-1f66-4726-a0ce-6b64e8c6d4cd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.260150 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b199a-1f66-4726-a0ce-6b64e8c6d4cd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.260432 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b199a-1f66-4726-a0ce-6b64e8c6d4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.311298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-714b199a-1f66-4726-a0ce-6b64e8c6d4cd None None] MFA Rules not processed for user `c178bb17bdbc4021bba9114784482c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:15.313256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-714b199a-1f66-4726-a0ce-6b64e8c6d4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:15.337609 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1235/2475] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:15.345934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d9546-3f3e-43ca-a7db-adc70a64195d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.346266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d9546-3f3e-43ca-a7db-adc70a64195d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.346619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d9546-3f3e-43ca-a7db-adc70a64195d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.368566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-724d9546-3f3e-43ca-a7db-adc70a64195d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:15.370112 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-724d9546-3f3e-43ca-a7db-adc70a64195d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:15.381664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1236/2476] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:15.386695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d9546-3f3e-43ca-a7db-adc70a64195d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.400120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d9546-3f3e-43ca-a7db-adc70a64195d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.406855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a099a2b-29c5-4b2a-86ff-2787d520c13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.407596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a099a2b-29c5-4b2a-86ff-2787d520c13c admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.409150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a099a2b-29c5-4b2a-86ff-2787d520c13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.409150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a099a2b-29c5-4b2a-86ff-2787d520c13c admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.409517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a099a2b-29c5-4b2a-86ff-2787d520c13c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:15.409765 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a099a2b-29c5-4b2a-86ff-2787d520c13c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.411473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a099a2b-29c5-4b2a-86ff-2787d520c13c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.427809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1237/2477] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:18:15.432970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a099a2b-29c5-4b2a-86ff-2787d520c13c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.441119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1241/2478] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:15.446448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a099a2b-29c5-4b2a-86ff-2787d520c13c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.447367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b199a-1f66-4726-a0ce-6b64e8c6d4cd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.452114 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f0fe7-4457-48de-88cc-16ae7bffa4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.452815 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f0fe7-4457-48de-88cc-16ae7bffa4de admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.453182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f0fe7-4457-48de-88cc-16ae7bffa4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.453503 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f0fe7-4457-48de-88cc-16ae7bffa4de admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.454376 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f0fe7-4457-48de-88cc-16ae7bffa4de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.457037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f0fe7-4457-48de-88cc-16ae7bffa4de admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.466094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b199a-1f66-4726-a0ce-6b64e8c6d4cd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.473569 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c66ee-767a-46fb-9605-23404ec18b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.474662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c66ee-767a-46fb-9605-23404ec18b91 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.476357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c66ee-767a-46fb-9605-23404ec18b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.476357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c66ee-767a-46fb-9605-23404ec18b91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.477316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c66ee-767a-46fb-9605-23404ec18b91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.480958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c66ee-767a-46fb-9605-23404ec18b91 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.510662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1238/2479] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:15 2022] POST /identity/v3/projects => generated 401 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:15.516763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f0fe7-4457-48de-88cc-16ae7bffa4de admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.526611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:15.527353 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1242/2480] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:15 2022] POST /identity/v3/projects => generated 405 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:15.532180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c66ee-767a-46fb-9605-23404ec18b91 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.574333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:15.617240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c66ee-767a-46fb-9605-23404ec18b91 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.624418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a7057-8c18-4b64-838a-95936fc8c016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.624985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a7057-8c18-4b64-838a-95936fc8c016 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.625346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a7057-8c18-4b64-838a-95936fc8c016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.625703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a7057-8c18-4b64-838a-95936fc8c016 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.626519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a7057-8c18-4b64-838a-95936fc8c016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.629215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a7057-8c18-4b64-838a-95936fc8c016 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.641388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f0fe7-4457-48de-88cc-16ae7bffa4de admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.652716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ccadf-8706-4f70-8d4d-1b31d4b78478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.653418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ccadf-8706-4f70-8d4d-1b31d4b78478 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.653707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ccadf-8706-4f70-8d4d-1b31d4b78478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.653983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ccadf-8706-4f70-8d4d-1b31d4b78478 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.654698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ccadf-8706-4f70-8d4d-1b31d4b78478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.656867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ccadf-8706-4f70-8d4d-1b31d4b78478 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.661491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1243/2481] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:15 2022] POST /identity/v3/users => generated 309 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:15.665862 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a7057-8c18-4b64-838a-95936fc8c016 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.679738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a7057-8c18-4b64-838a-95936fc8c016 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.685463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dee58-2c7f-4447-9381-88846c596685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.686287 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dee58-2c7f-4447-9381-88846c596685 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.686658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dee58-2c7f-4447-9381-88846c596685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.687008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dee58-2c7f-4447-9381-88846c596685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.687923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dee58-2c7f-4447-9381-88846c596685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.689658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dee58-2c7f-4447-9381-88846c596685 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.690893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1239/2482] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:15 2022] POST /identity/v3/users => generated 301 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:15.695188 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ccadf-8706-4f70-8d4d-1b31d4b78478 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.709121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1244/2483] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:15.711832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ccadf-8706-4f70-8d4d-1b31d4b78478 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.713953 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dee58-2c7f-4447-9381-88846c596685 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.717337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02214213-2c77-42f9-bae8-4c1c48bf9113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.718031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02214213-2c77-42f9-bae8-4c1c48bf9113 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.718320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02214213-2c77-42f9-bae8-4c1c48bf9113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.718600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02214213-2c77-42f9-bae8-4c1c48bf9113 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.719342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02214213-2c77-42f9-bae8-4c1c48bf9113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.721043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02214213-2c77-42f9-bae8-4c1c48bf9113 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.738337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1240/2484] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:15.741333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dee58-2c7f-4447-9381-88846c596685 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.744297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02214213-2c77-42f9-bae8-4c1c48bf9113 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.754504 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90874b7b-0c9c-4ec2-8b76-3ac3056fd026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.755906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90874b7b-0c9c-4ec2-8b76-3ac3056fd026 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.756461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90874b7b-0c9c-4ec2-8b76-3ac3056fd026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.756839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02214213-2c77-42f9-bae8-4c1c48bf9113 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.757362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90874b7b-0c9c-4ec2-8b76-3ac3056fd026 admin admin] PATH_INFO: `/v3/projects/6d9f05c4e6c74b7582ef466d13751144/users/6c5f3151dde54641a242f19283d21adb/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.764492 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713b259-4566-4be5-8f01-3dfb9e8fdcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.764804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b259-4566-4be5-8f01-3dfb9e8fdcc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.765078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b259-4566-4be5-8f01-3dfb9e8fdcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.765360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b259-4566-4be5-8f01-3dfb9e8fdcc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.766176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713b259-4566-4be5-8f01-3dfb9e8fdcc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.767921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713b259-4566-4be5-8f01-3dfb9e8fdcc1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.781120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90874b7b-0c9c-4ec2-8b76-3ac3056fd026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d9f05c4e6c74b7582ef466d13751144, user_id=6c5f3151dde54641a242f19283d21adb, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.784508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90874b7b-0c9c-4ec2-8b76-3ac3056fd026 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.788499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1241/2485] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:15.799136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713b259-4566-4be5-8f01-3dfb9e8fdcc1 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.801589 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1245/2486] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:15 2022] PUT /identity/v3/projects/6d9f05c4e6c74b7582ef466d13751144/users/6c5f3151dde54641a242f19283d21adb/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:15.805758 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90874b7b-0c9c-4ec2-8b76-3ac3056fd026 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.813764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713b259-4566-4be5-8f01-3dfb9e8fdcc1 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.887568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1cf4a-5167-434d-b0e2-7db814f3b7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.888711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1cf4a-5167-434d-b0e2-7db814f3b7c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.889182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1cf4a-5167-434d-b0e2-7db814f3b7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.889628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1cf4a-5167-434d-b0e2-7db814f3b7c0 admin admin] PATH_INFO: `/v3/projects/cbe26d73033a44f98d3d638d584aa139/users/740b427344a741eeb00b764b36ddc223/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.897920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90874b7b-0c9c-4ec2-8b76-3ac3056fd026 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.903429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446d894-5757-435b-b2f6-ee8c7578f3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:15.904383 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446d894-5757-435b-b2f6-ee8c7578f3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.904743 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446d894-5757-435b-b2f6-ee8c7578f3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.905101 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446d894-5757-435b-b2f6-ee8c7578f3d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.905942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446d894-5757-435b-b2f6-ee8c7578f3d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.907728 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446d894-5757-435b-b2f6-ee8c7578f3d4 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.912905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1cf4a-5167-434d-b0e2-7db814f3b7c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe26d73033a44f98d3d638d584aa139, user_id=740b427344a741eeb00b764b36ddc223, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:15.916253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1cf4a-5167-434d-b0e2-7db814f3b7c0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:15.924085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1246/2487] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:15.925962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446d894-5757-435b-b2f6-ee8c7578f3d4 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:15.938950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446d894-5757-435b-b2f6-ee8c7578f3d4 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:15.939639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1242/2488] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:15 2022] PUT /identity/v3/projects/cbe26d73033a44f98d3d638d584aa139/users/740b427344a741eeb00b764b36ddc223/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:15.946479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a6059-666d-488d-83dc-02f99c856779 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:15.946781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a6059-666d-488d-83dc-02f99c856779 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:15.947075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a6059-666d-488d-83dc-02f99c856779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:15.970211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-048a6059-666d-488d-83dc-02f99c856779 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:15.972162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-048a6059-666d-488d-83dc-02f99c856779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:16.042712 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:16.060376 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4cf52-9799-4c2b-b4af-6531b400b927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:16.061775 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4cf52-9799-4c2b-b4af-6531b400b927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:16.062296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4cf52-9799-4c2b-b4af-6531b400b927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:16.063310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4cf52-9799-4c2b-b4af-6531b400b927 admin admin] PATH_INFO: `/v3/projects/6d9f05c4e6c74b7582ef466d13751144/users/6c5f3151dde54641a242f19283d21adb/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:16.068939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1243/2489] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:16.069233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4cf52-9799-4c2b-b4af-6531b400b927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d9f05c4e6c74b7582ef466d13751144, user_id=6c5f3151dde54641a242f19283d21adb, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:16.072242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4cf52-9799-4c2b-b4af-6531b400b927 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:16.078233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a6059-666d-488d-83dc-02f99c856779 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:16.081561 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-048a6059-666d-488d-83dc-02f99c856779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:16.099192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1247/2490] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:15 2022] PUT /identity/v3/projects/6d9f05c4e6c74b7582ef466d13751144/users/6c5f3151dde54641a242f19283d21adb/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:16.108755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57e532-685b-4777-94a1-942b3957b52e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:16.109500 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57e532-685b-4777-94a1-942b3957b52e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:16.110024 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57e532-685b-4777-94a1-942b3957b52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:16.165365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a6059-666d-488d-83dc-02f99c856779 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:16.167435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-aa57e532-685b-4777-94a1-942b3957b52e None None] MFA Rules not processed for user `6c5f3151dde54641a242f19283d21adb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:16.169213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-aa57e532-685b-4777-94a1-942b3957b52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:16.179579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:16.260444 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfa116-c3bb-4623-91d4-01b975383778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:16.261496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfa116-c3bb-4623-91d4-01b975383778 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:16.261827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfa116-c3bb-4623-91d4-01b975383778 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:16.262179 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfa116-c3bb-4623-91d4-01b975383778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:16.313695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1248/2491] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:16 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:16.332855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57e532-685b-4777-94a1-942b3957b52e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:16.335371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-aa57e532-685b-4777-94a1-942b3957b52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:16.382246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfa116-c3bb-4623-91d4-01b975383778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:16.384160 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfa116-c3bb-4623-91d4-01b975383778 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:16.404162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1244/2492] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:16.429089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57e532-685b-4777-94a1-942b3957b52e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:16.435699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf428141-1c03-450a-b5e1-bcb9377541c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:16.436865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf428141-1c03-450a-b5e1-bcb9377541c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:16.437367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf428141-1c03-450a-b5e1-bcb9377541c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:16.437865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf428141-1c03-450a-b5e1-bcb9377541c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:16.458815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf428141-1c03-450a-b5e1-bcb9377541c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:16.462157 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf428141-1c03-450a-b5e1-bcb9377541c5 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:16.482857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1249/2493] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:16 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:18:16.755602 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfa116-c3bb-4623-91d4-01b975383778 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:16.798588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfa116-c3bb-4623-91d4-01b975383778 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:16.822572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:16.824441 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:16.824962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:16.825572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] PATH_INFO: `/v3/users/c178bb17bdbc4021bba9114784482c58` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:16.839041 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c178bb17bdbc4021bba9114784482c58)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:16.842858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:16.977931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c178bb17bdbc4021bba9114784482c58'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:16.995097 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c178bb17bdbc4021bba9114784482c58'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:16.996973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c178bb17bdbc4021bba9114784482c58'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:17.000008 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:17.018814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c178bb17bdbc4021bba9114784482c58'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:17.020364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c178bb17bdbc4021bba9114784482c58'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:17.061183 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1245/2494] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:16 2022] DELETE /identity/v3/users/c178bb17bdbc4021bba9114784482c58 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:17.070350 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aebafa-cd6b-47d0-bb29-9227ba821aef None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:17.070989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aebafa-cd6b-47d0-bb29-9227ba821aef None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:17.071916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aebafa-cd6b-47d0-bb29-9227ba821aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:17.107161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-b6aebafa-cd6b-47d0-bb29-9227ba821aef None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:17.109595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b6aebafa-cd6b-47d0-bb29-9227ba821aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:17.245812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1250/2495] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:17.260421 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:17.262631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:17.363820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05615a33-4d79-450b-afdb-1a64eaeae65a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:17.370354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebbc1c-2480-4937-86dc-31b3a5ddce9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:17.371559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebbc1c-2480-4937-86dc-31b3a5ddce9f service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:17.372073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebbc1c-2480-4937-86dc-31b3a5ddce9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:17.372596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebbc1c-2480-4937-86dc-31b3a5ddce9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:17.391059 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebbc1c-2480-4937-86dc-31b3a5ddce9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:17.394614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebbc1c-2480-4937-86dc-31b3a5ddce9f service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:17.414528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1246/2496] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:17.834687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aebafa-cd6b-47d0-bb29-9227ba821aef None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:17.837165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b6aebafa-cd6b-47d0-bb29-9227ba821aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:17.863882 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aebafa-cd6b-47d0-bb29-9227ba821aef None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:17.870442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26429b-1a34-4cc5-9693-a2542a5af552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:17.871537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26429b-1a34-4cc5-9693-a2542a5af552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:17.871984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26429b-1a34-4cc5-9693-a2542a5af552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:17.872394 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26429b-1a34-4cc5-9693-a2542a5af552 admin admin] PATH_INFO: `/v3/projects/3d52af11ee044ecfb62858553f57d3f3` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:17.874107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26429b-1a34-4cc5-9693-a2542a5af552 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d52af11ee044ecfb62858553f57d3f3)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:17.877206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26429b-1a34-4cc5-9693-a2542a5af552 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:17.942709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:17.955200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9c26429b-1a34-4cc5-9693-a2542a5af552 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d52af11ee044ecfb62858553f57d3f3', 'request_id': 'req-9c26429b-1a34-4cc5-9693-a2542a5af552'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:17.965029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9c26429b-1a34-4cc5-9693-a2542a5af552 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d52af11ee044ecfb62858553f57d3f3', 'request_id': 'req-9c26429b-1a34-4cc5-9693-a2542a5af552'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:17.973374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9c26429b-1a34-4cc5-9693-a2542a5af552 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d52af11ee044ecfb62858553f57d3f3', 'request_id': 'req-9c26429b-1a34-4cc5-9693-a2542a5af552'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:18.000767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9c26429b-1a34-4cc5-9693-a2542a5af552 admin admin] The token cache is being invalidate because project 3d52af11ee044ecfb62858553f57d3f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:18.001287 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9c26429b-1a34-4cc5-9693-a2542a5af552 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:18.003468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-9c26429b-1a34-4cc5-9693-a2542a5af552 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:18.006329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1251/2497] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:17 2022] DELETE /identity/v3/projects/3d52af11ee044ecfb62858553f57d3f3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:18.012700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebbc1c-2480-4937-86dc-31b3a5ddce9f service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:18.054987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebbc1c-2480-4937-86dc-31b3a5ddce9f service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:18.201263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:18.203636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:18.204509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:18.205472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] PATH_INFO: `/v3/users/6c5f3151dde54641a242f19283d21adb` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:18.215096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c5f3151dde54641a242f19283d21adb)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:18.219203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:18.265042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8805fd-1732-4920-b049-bb45d386d2ad None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:18.265623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8805fd-1732-4920-b049-bb45d386d2ad None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:18.265971 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8805fd-1732-4920-b049-bb45d386d2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:18.322313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3d8805fd-1732-4920-b049-bb45d386d2ad None None] MFA Rules not processed for user `740b427344a741eeb00b764b36ddc223`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:18.324513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3d8805fd-1732-4920-b049-bb45d386d2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:18.361117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c5f3151dde54641a242f19283d21adb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:18.387097 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c5f3151dde54641a242f19283d21adb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:18.387692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c5f3151dde54641a242f19283d21adb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:18.406140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c5f3151dde54641a242f19283d21adb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:18.410052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c5f3151dde54641a242f19283d21adb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:18.446067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:18.476926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1247/2498] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:18 2022] DELETE /identity/v3/users/6c5f3151dde54641a242f19283d21adb => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:18.488419 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1252/2499] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:18.493015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:18.494837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:18.656898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873da9c0-3688-4004-893b-2a7807f963e2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:18.680060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caae71d-441e-42b6-8cf4-252dee33b426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:18.682019 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caae71d-441e-42b6-8cf4-252dee33b426 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:18.682906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caae71d-441e-42b6-8cf4-252dee33b426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:18.683708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caae71d-441e-42b6-8cf4-252dee33b426 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:18.685362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caae71d-441e-42b6-8cf4-252dee33b426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:18.692742 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caae71d-441e-42b6-8cf4-252dee33b426 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:18.780494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1248/2500] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:18 2022] POST /identity/v3/projects => generated 405 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:18.786586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8805fd-1732-4920-b049-bb45d386d2ad None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:18.813486 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8805fd-1732-4920-b049-bb45d386d2ad None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:18.977135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a502ad-784a-47bb-95f7-026fb7181b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:18.978328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a502ad-784a-47bb-95f7-026fb7181b47 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:18.979076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a502ad-784a-47bb-95f7-026fb7181b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:18.979076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a502ad-784a-47bb-95f7-026fb7181b47 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:18.980063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a502ad-784a-47bb-95f7-026fb7181b47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:18.982893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a502ad-784a-47bb-95f7-026fb7181b47 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.017580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caae71d-441e-42b6-8cf4-252dee33b426 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.020351 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-1caae71d-441e-42b6-8cf4-252dee33b426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:19.041857 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1253/2501] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:18 2022] POST /identity/v3/users => generated 309 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:19.043766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a502ad-784a-47bb-95f7-026fb7181b47 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.049020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caae71d-441e-42b6-8cf4-252dee33b426 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.062565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137b00d-0767-4141-946a-9aaffd5abab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.064532 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137b00d-0767-4141-946a-9aaffd5abab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.066064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137b00d-0767-4141-946a-9aaffd5abab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.066546 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137b00d-0767-4141-946a-9aaffd5abab7 admin admin] PATH_INFO: `/v3/projects/6d9f05c4e6c74b7582ef466d13751144` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.067263 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a502ad-784a-47bb-95f7-026fb7181b47 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.068944 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137b00d-0767-4141-946a-9aaffd5abab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d9f05c4e6c74b7582ef466d13751144)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.073312 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137b00d-0767-4141-946a-9aaffd5abab7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.077823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7a054-697b-4417-b2b7-6cee46959386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.078796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7a054-697b-4417-b2b7-6cee46959386 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.079258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7a054-697b-4417-b2b7-6cee46959386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.079828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7a054-697b-4417-b2b7-6cee46959386 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.081489 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7a054-697b-4417-b2b7-6cee46959386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.085726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7a054-697b-4417-b2b7-6cee46959386 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.103317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1254/2502] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:19.111496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7a054-697b-4417-b2b7-6cee46959386 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.126550 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7a054-697b-4417-b2b7-6cee46959386 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.135791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84706a-2dbd-4aae-b22c-8103ef9720a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.137370 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84706a-2dbd-4aae-b22c-8103ef9720a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.137797 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84706a-2dbd-4aae-b22c-8103ef9720a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.138262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84706a-2dbd-4aae-b22c-8103ef9720a3 admin admin] PATH_INFO: `/v3/projects/b31c6f6aabc546d5b178e799c788200d/users/47da623af8ea4eb5bed4aad9e730a0a4/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.165331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84706a-2dbd-4aae-b22c-8103ef9720a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31c6f6aabc546d5b178e799c788200d, user_id=47da623af8ea4eb5bed4aad9e730a0a4, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.169243 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84706a-2dbd-4aae-b22c-8103ef9720a3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.195599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1255/2503] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:19 2022] PUT /identity/v3/projects/b31c6f6aabc546d5b178e799c788200d/users/47da623af8ea4eb5bed4aad9e730a0a4/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:19.201014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84706a-2dbd-4aae-b22c-8103ef9720a3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.206683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e137b00d-0767-4141-946a-9aaffd5abab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d9f05c4e6c74b7582ef466d13751144', 'request_id': 'req-e137b00d-0767-4141-946a-9aaffd5abab7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:19.228162 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:19.240053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e137b00d-0767-4141-946a-9aaffd5abab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d9f05c4e6c74b7582ef466d13751144', 'request_id': 'req-e137b00d-0767-4141-946a-9aaffd5abab7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:19.257619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e137b00d-0767-4141-946a-9aaffd5abab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d9f05c4e6c74b7582ef466d13751144', 'request_id': 'req-e137b00d-0767-4141-946a-9aaffd5abab7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:19.282449 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e137b00d-0767-4141-946a-9aaffd5abab7 admin admin] The token cache is being invalidate because project 6d9f05c4e6c74b7582ef466d13751144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:19.282956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e137b00d-0767-4141-946a-9aaffd5abab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:19.284770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e137b00d-0767-4141-946a-9aaffd5abab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:19.294644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1249/2504] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:19 2022] DELETE /identity/v3/projects/6d9f05c4e6c74b7582ef466d13751144 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:19.297482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e87a8-a8eb-47ca-833e-a120e00e44b6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.297994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e87a8-a8eb-47ca-833e-a120e00e44b6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.298588 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e87a8-a8eb-47ca-833e-a120e00e44b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.331574 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84706a-2dbd-4aae-b22c-8103ef9720a3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.334355 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bb4e87a8-a8eb-47ca-833e-a120e00e44b6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:19.336795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bb4e87a8-a8eb-47ca-833e-a120e00e44b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:19.341570 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d6825-81d1-4719-aec5-21835595c574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.342468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6825-81d1-4719-aec5-21835595c574 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.342883 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6825-81d1-4719-aec5-21835595c574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.343309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6825-81d1-4719-aec5-21835595c574 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.344169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d6825-81d1-4719-aec5-21835595c574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.347365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d6825-81d1-4719-aec5-21835595c574 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.352146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1250/2505] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:19.363206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1256/2506] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:19.364910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2b692-6815-4273-8a10-dfa7fbb43871 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.365400 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2b692-6815-4273-8a10-dfa7fbb43871 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.365896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2b692-6815-4273-8a10-dfa7fbb43871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.372481 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d6825-81d1-4719-aec5-21835595c574 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.374554 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b58d6825-81d1-4719-aec5-21835595c574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:19.390085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d6825-81d1-4719-aec5-21835595c574 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.398314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a587b-b45b-4a63-80c8-ad28e1c888d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.399418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a587b-b45b-4a63-80c8-ad28e1c888d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.399795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a587b-b45b-4a63-80c8-ad28e1c888d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.400192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a587b-b45b-4a63-80c8-ad28e1c888d3 admin admin] PATH_INFO: `/v3/projects/b31c6f6aabc546d5b178e799c788200d/users/47da623af8ea4eb5bed4aad9e730a0a4/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.402521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a587b-b45b-4a63-80c8-ad28e1c888d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31c6f6aabc546d5b178e799c788200d, user_id=47da623af8ea4eb5bed4aad9e730a0a4, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.404839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-17d2b692-6815-4273-8a10-dfa7fbb43871 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:19.406375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a587b-b45b-4a63-80c8-ad28e1c888d3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.406732 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-17d2b692-6815-4273-8a10-dfa7fbb43871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:19.418962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1251/2507] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:19.421702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1257/2508] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:19 2022] PUT /identity/v3/projects/b31c6f6aabc546d5b178e799c788200d/users/47da623af8ea4eb5bed4aad9e730a0a4/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:19.426012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2b692-6815-4273-8a10-dfa7fbb43871 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.443802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2b692-6815-4273-8a10-dfa7fbb43871 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.527534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:19.561099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bf4d1-769c-44a6-9088-24251fa2d441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.562095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bf4d1-769c-44a6-9088-24251fa2d441 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.562303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bf4d1-769c-44a6-9088-24251fa2d441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.562992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bf4d1-769c-44a6-9088-24251fa2d441 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.563627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bf4d1-769c-44a6-9088-24251fa2d441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:19.564436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bf4d1-769c-44a6-9088-24251fa2d441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.566772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bf4d1-769c-44a6-9088-24251fa2d441 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.588041 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1252/2509] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:18:19.593407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a587b-b45b-4a63-80c8-ad28e1c888d3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.614780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a587b-b45b-4a63-80c8-ad28e1c888d3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.622801 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64165de-6780-42e5-a326-7dfa8dc07b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.623725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64165de-6780-42e5-a326-7dfa8dc07b7c admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.623999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64165de-6780-42e5-a326-7dfa8dc07b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.624302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64165de-6780-42e5-a326-7dfa8dc07b7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.624994 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64165de-6780-42e5-a326-7dfa8dc07b7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.627016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64165de-6780-42e5-a326-7dfa8dc07b7c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.689544 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1258/2510] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:19 2022] POST /identity/v3/projects => generated 357 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:19.694513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bf4d1-769c-44a6-9088-24251fa2d441 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.710807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bf4d1-769c-44a6-9088-24251fa2d441 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.835675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d2cf8-4f3c-4f40-aa90-7ae03fc54bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.837178 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d2cf8-4f3c-4f40-aa90-7ae03fc54bea admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.837736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d2cf8-4f3c-4f40-aa90-7ae03fc54bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.838240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d2cf8-4f3c-4f40-aa90-7ae03fc54bea admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.840917 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d2cf8-4f3c-4f40-aa90-7ae03fc54bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.845488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d2cf8-4f3c-4f40-aa90-7ae03fc54bea admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.908807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1253/2511] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:19 2022] POST /identity/v3/users => generated 279 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:19.916188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64165de-6780-42e5-a326-7dfa8dc07b7c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:19.938147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64165de-6780-42e5-a326-7dfa8dc07b7c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:19.951536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b57914-a3f4-41eb-9b02-99a8e8b96cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:19.952495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b57914-a3f4-41eb-9b02-99a8e8b96cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:19.965027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b57914-a3f4-41eb-9b02-99a8e8b96cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:19.965317 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b57914-a3f4-41eb-9b02-99a8e8b96cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:19.966872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b57914-a3f4-41eb-9b02-99a8e8b96cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:19.969801 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b57914-a3f4-41eb-9b02-99a8e8b96cf8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:19.993661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1259/2512] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:19.995356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d2cf8-4f3c-4f40-aa90-7ae03fc54bea admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:20.015758 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d2cf8-4f3c-4f40-aa90-7ae03fc54bea admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:20.025602 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85138b-6ee8-45eb-819a-001181e09dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:20.028287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85138b-6ee8-45eb-819a-001181e09dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:20.028842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85138b-6ee8-45eb-819a-001181e09dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:20.029538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85138b-6ee8-45eb-819a-001181e09dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:20.030983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85138b-6ee8-45eb-819a-001181e09dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:20.035088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85138b-6ee8-45eb-819a-001181e09dc6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:20.069876 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1254/2513] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:20.075713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b57914-a3f4-41eb-9b02-99a8e8b96cf8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:20.092362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b57914-a3f4-41eb-9b02-99a8e8b96cf8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:20.100954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfc722-5cc2-4495-8144-819e359db602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:20.102858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfc722-5cc2-4495-8144-819e359db602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:20.103282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfc722-5cc2-4495-8144-819e359db602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:20.103928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfc722-5cc2-4495-8144-819e359db602 admin admin] PATH_INFO: `/v3/projects/12257ef0c75c447c8dc88488fe119db3/users/2d32aa78535b4dbb8bbc15139576284a/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:20.128097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfc722-5cc2-4495-8144-819e359db602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12257ef0c75c447c8dc88488fe119db3, user_id=2d32aa78535b4dbb8bbc15139576284a, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:20.131485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfc722-5cc2-4495-8144-819e359db602 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:20.151902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1260/2514] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:20 2022] PUT /identity/v3/projects/12257ef0c75c447c8dc88488fe119db3/users/2d32aa78535b4dbb8bbc15139576284a/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:20.164223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba156cc-c81e-4ef0-b5fe-cb7f723fa9be None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:20.164518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba156cc-c81e-4ef0-b5fe-cb7f723fa9be None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:20.165016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba156cc-c81e-4ef0-b5fe-cb7f723fa9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:20.196676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9ba156cc-c81e-4ef0-b5fe-cb7f723fa9be None None] MFA Rules not processed for user `2d32aa78535b4dbb8bbc15139576284a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:20.198288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9ba156cc-c81e-4ef0-b5fe-cb7f723fa9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:20.320556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1255/2515] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:20 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:20.325923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfc722-5cc2-4495-8144-819e359db602 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:20.422215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfc722-5cc2-4495-8144-819e359db602 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:20.428541 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f7ab4-19a7-452d-9ccd-5812aa0a04c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:20.429074 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f7ab4-19a7-452d-9ccd-5812aa0a04c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:20.429778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f7ab4-19a7-452d-9ccd-5812aa0a04c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:20.430082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f7ab4-19a7-452d-9ccd-5812aa0a04c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:20.430985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f7ab4-19a7-452d-9ccd-5812aa0a04c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:20.433091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f7ab4-19a7-452d-9ccd-5812aa0a04c2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:20.473146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1261/2516] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:20 2022] POST /identity/v3/projects => generated 357 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:20.478074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba156cc-c81e-4ef0-b5fe-cb7f723fa9be None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:20.495769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba156cc-c81e-4ef0-b5fe-cb7f723fa9be None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:20.571004 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:20.589012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947deae7-489b-4041-99ec-dd06e8748765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:20.590203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947deae7-489b-4041-99ec-dd06e8748765 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:20.593871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947deae7-489b-4041-99ec-dd06e8748765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:20.594424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947deae7-489b-4041-99ec-dd06e8748765 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:20.595349 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947deae7-489b-4041-99ec-dd06e8748765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:20.597664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947deae7-489b-4041-99ec-dd06e8748765 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:20.625538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1256/2517] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:20 2022] POST /identity/v3/users => generated 285 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:20.628421 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f7ab4-19a7-452d-9ccd-5812aa0a04c2 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:20.653970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f7ab4-19a7-452d-9ccd-5812aa0a04c2 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:20.663501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c76707-232f-4b07-b87d-ee78d2acda7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:20.665168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c76707-232f-4b07-b87d-ee78d2acda7c admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:20.665168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c76707-232f-4b07-b87d-ee78d2acda7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:20.665673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c76707-232f-4b07-b87d-ee78d2acda7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:20.667106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c76707-232f-4b07-b87d-ee78d2acda7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:20.670441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c76707-232f-4b07-b87d-ee78d2acda7c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:20.695601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1262/2518] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:20.700458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947deae7-489b-4041-99ec-dd06e8748765 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:20.727903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947deae7-489b-4041-99ec-dd06e8748765 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:20.738682 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a2057-c1ac-4fbf-b3cc-aa8daffecc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:20.739710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a2057-c1ac-4fbf-b3cc-aa8daffecc88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:20.740007 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a2057-c1ac-4fbf-b3cc-aa8daffecc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:20.740347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a2057-c1ac-4fbf-b3cc-aa8daffecc88 admin admin] PATH_INFO: `/v3/projects/50e5302ec83844f080a0ac1ed592a617/users/b381872678754ca3ae6ff11a06ad9e3c/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:20.763840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a2057-c1ac-4fbf-b3cc-aa8daffecc88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e5302ec83844f080a0ac1ed592a617, user_id=b381872678754ca3ae6ff11a06ad9e3c, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:20.767956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a2057-c1ac-4fbf-b3cc-aa8daffecc88 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:20.798287 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1257/2519] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:20 2022] PUT /identity/v3/projects/50e5302ec83844f080a0ac1ed592a617/users/b381872678754ca3ae6ff11a06ad9e3c/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:20.804552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c76707-232f-4b07-b87d-ee78d2acda7c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:20.843307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c76707-232f-4b07-b87d-ee78d2acda7c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:20.859761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:20.976906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2903b-647c-451a-b061-704ab3e17be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:20.977637 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2903b-647c-451a-b061-704ab3e17be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:20.977947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2903b-647c-451a-b061-704ab3e17be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:20.978265 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2903b-647c-451a-b061-704ab3e17be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:20.979145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2903b-647c-451a-b061-704ab3e17be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:20.981057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2903b-647c-451a-b061-704ab3e17be1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:20.999938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1263/2520] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:21.005225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a2057-c1ac-4fbf-b3cc-aa8daffecc88 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:21.018354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a2057-c1ac-4fbf-b3cc-aa8daffecc88 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:21.023630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241e0fb-f988-4456-b54b-cccdf8576344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:21.024932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241e0fb-f988-4456-b54b-cccdf8576344 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:21.025454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241e0fb-f988-4456-b54b-cccdf8576344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:21.025958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241e0fb-f988-4456-b54b-cccdf8576344 admin admin] PATH_INFO: `/v3/projects/50e5302ec83844f080a0ac1ed592a617/users/b381872678754ca3ae6ff11a06ad9e3c/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:21.028583 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241e0fb-f988-4456-b54b-cccdf8576344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e5302ec83844f080a0ac1ed592a617, user_id=b381872678754ca3ae6ff11a06ad9e3c, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:21.035362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241e0fb-f988-4456-b54b-cccdf8576344 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:21.045073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1258/2521] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:21 2022] PUT /identity/v3/projects/50e5302ec83844f080a0ac1ed592a617/users/b381872678754ca3ae6ff11a06ad9e3c/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:21.057913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2254a-f54e-4141-bc60-5a1b51b79d13 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:21.058273 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2254a-f54e-4141-bc60-5a1b51b79d13 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:21.058480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2254a-f54e-4141-bc60-5a1b51b79d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:21.095231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-fbe2254a-f54e-4141-bc60-5a1b51b79d13 None None] MFA Rules not processed for user `b381872678754ca3ae6ff11a06ad9e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:21.097213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fbe2254a-f54e-4141-bc60-5a1b51b79d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:21.155155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8e931-aedb-4b39-8b56-422cc5fab08d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:21.155904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8e931-aedb-4b39-8b56-422cc5fab08d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:21.156362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8e931-aedb-4b39-8b56-422cc5fab08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:21.194412 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-39e8e931-aedb-4b39-8b56-422cc5fab08d None None] MFA Rules not processed for user `47da623af8ea4eb5bed4aad9e730a0a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:21.195709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-39e8e931-aedb-4b39-8b56-422cc5fab08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:21.231666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1264/2522] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:21 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:21.243676 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2254a-f54e-4141-bc60-5a1b51b79d13 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:21.245526 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fbe2254a-f54e-4141-bc60-5a1b51b79d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:21.309982 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1259/2523] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:21 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:21.336945 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2254a-f54e-4141-bc60-5a1b51b79d13 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:21.345940 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56d963-7408-4283-8436-b568c10839f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:21.346661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56d963-7408-4283-8436-b568c10839f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:21.346948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56d963-7408-4283-8436-b568c10839f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:21.350383 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56d963-7408-4283-8436-b568c10839f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:21.371011 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56d963-7408-4283-8436-b568c10839f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:21.372554 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56d963-7408-4283-8436-b568c10839f9 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:21.385400 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1265/2524] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:21 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:21.446699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8e931-aedb-4b39-8b56-422cc5fab08d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:21.448675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-39e8e931-aedb-4b39-8b56-422cc5fab08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:21.516702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56d963-7408-4283-8436-b568c10839f9 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:21.529016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56d963-7408-4283-8436-b568c10839f9 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:21.535470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3d74c-78f6-48d5-b1b3-a84591f99a88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:21.536307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3d74c-78f6-48d5-b1b3-a84591f99a88 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:21.536686 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3d74c-78f6-48d5-b1b3-a84591f99a88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:21.537111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3d74c-78f6-48d5-b1b3-a84591f99a88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:21.548281 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8e931-aedb-4b39-8b56-422cc5fab08d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:21.554695 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:21.555645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:21.555934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:21.556272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] PATH_INFO: `/v3/users/bf3817e18de149aba43a2f65f37d63f4` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:21.557739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf3817e18de149aba43a2f65f37d63f4)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:21.560892 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:21.609041 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:21.612684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:21.648614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3d74c-78f6-48d5-b1b3-a84591f99a88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:21.651277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3d74c-78f6-48d5-b1b3-a84591f99a88 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:21.656557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf3817e18de149aba43a2f65f37d63f4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:21.669811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1266/2525] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:21 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:18:21.675563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf3817e18de149aba43a2f65f37d63f4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:21.676488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf3817e18de149aba43a2f65f37d63f4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:21.695087 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf3817e18de149aba43a2f65f37d63f4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:21.696458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf3817e18de149aba43a2f65f37d63f4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:21.730363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1260/2526] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:21 2022] DELETE /identity/v3/users/bf3817e18de149aba43a2f65f37d63f4 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:22.024509 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3d74c-78f6-48d5-b1b3-a84591f99a88 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:22.024858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:22.026836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-43a3d74c-78f6-48d5-b1b3-a84591f99a88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:22.027375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:22.168796 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae4334-38ee-4e73-b4c0-cfa23ff52d56 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:22.177313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37ff32-aa68-429d-bce5-7c1981b12ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:22.178647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3d74c-78f6-48d5-b1b3-a84591f99a88 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:22.179473 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37ff32-aa68-429d-bce5-7c1981b12ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:22.180465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37ff32-aa68-429d-bce5-7c1981b12ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:22.181088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37ff32-aa68-429d-bce5-7c1981b12ed2 admin admin] PATH_INFO: `/v3/projects/9b56ea0be2964e21985deb5964232923` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:22.182934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37ff32-aa68-429d-bce5-7c1981b12ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b56ea0be2964e21985deb5964232923)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:22.186204 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37ff32-aa68-429d-bce5-7c1981b12ed2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:22.275818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-be37ff32-aa68-429d-bce5-7c1981b12ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b56ea0be2964e21985deb5964232923', 'request_id': 'req-be37ff32-aa68-429d-bce5-7c1981b12ed2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:22.278515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ab725-19eb-4e91-9bc1-3a09cb826f08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:22.279503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ab725-19eb-4e91-9bc1-3a09cb826f08 service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:22.280022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ab725-19eb-4e91-9bc1-3a09cb826f08 service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:22.280780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ab725-19eb-4e91-9bc1-3a09cb826f08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:22.283965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-462ab725-19eb-4e91-9bc1-3a09cb826f08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:22.284263 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-be37ff32-aa68-429d-bce5-7c1981b12ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b56ea0be2964e21985deb5964232923', 'request_id': 'req-be37ff32-aa68-429d-bce5-7c1981b12ed2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:22.292031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-be37ff32-aa68-429d-bce5-7c1981b12ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b56ea0be2964e21985deb5964232923', 'request_id': 'req-be37ff32-aa68-429d-bce5-7c1981b12ed2'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:22.303337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ab725-19eb-4e91-9bc1-3a09cb826f08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:22.306302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ab725-19eb-4e91-9bc1-3a09cb826f08 service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:22.309793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-be37ff32-aa68-429d-bce5-7c1981b12ed2 admin admin] The token cache is being invalidate because project 9b56ea0be2964e21985deb5964232923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:22.309793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-be37ff32-aa68-429d-bce5-7c1981b12ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:22.310563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-be37ff32-aa68-429d-bce5-7c1981b12ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:22.312818 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1261/2527] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:22 2022] DELETE /identity/v3/projects/9b56ea0be2964e21985deb5964232923 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:22.318472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e9026-6f05-45ef-a8f8-49f88cb10b46 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:22.318929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e9026-6f05-45ef-a8f8-49f88cb10b46 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:22.319320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e9026-6f05-45ef-a8f8-49f88cb10b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:22.322316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1267/2528] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:22.341917 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9d7e9026-6f05-45ef-a8f8-49f88cb10b46 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:22.344180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9d7e9026-6f05-45ef-a8f8-49f88cb10b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:22.398377 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:22.442143 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1262/2529] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:22.451651 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2bae2-284f-4e21-9ca0-57dfcf9163fb None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:22.452123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2bae2-284f-4e21-9ca0-57dfcf9163fb None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:22.452480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2bae2-284f-4e21-9ca0-57dfcf9163fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:22.478934 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-37b2bae2-284f-4e21-9ca0-57dfcf9163fb None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:22.480034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-37b2bae2-284f-4e21-9ca0-57dfcf9163fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:22.492584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1268/2530] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:22.494328 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e9026-6f05-45ef-a8f8-49f88cb10b46 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:22.513605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e9026-6f05-45ef-a8f8-49f88cb10b46 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:22.521301 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2d34c-e38b-411b-8054-85ba113d27f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:22.522012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2d34c-e38b-411b-8054-85ba113d27f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:22.522359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2d34c-e38b-411b-8054-85ba113d27f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:22.522585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2d34c-e38b-411b-8054-85ba113d27f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:22.523371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2d34c-e38b-411b-8054-85ba113d27f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:22.523711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2d34c-e38b-411b-8054-85ba113d27f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:22.525404 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2d34c-e38b-411b-8054-85ba113d27f4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:22.540128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1263/2531] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:18:22.545313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2bae2-284f-4e21-9ca0-57dfcf9163fb None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:22.563455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2bae2-284f-4e21-9ca0-57dfcf9163fb None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:22.572493 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66837a33-a977-4276-9790-70a1d5add03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:22.573748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66837a33-a977-4276-9790-70a1d5add03f admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:22.574321 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66837a33-a977-4276-9790-70a1d5add03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:22.574909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66837a33-a977-4276-9790-70a1d5add03f admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:22.576077 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66837a33-a977-4276-9790-70a1d5add03f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:22.579080 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66837a33-a977-4276-9790-70a1d5add03f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:22.625121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1269/2532] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:22 2022] POST /identity/v3/projects => generated 367 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:22.631010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2d34c-e38b-411b-8054-85ba113d27f4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:22.644745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2d34c-e38b-411b-8054-85ba113d27f4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:22.875328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66837a33-a977-4276-9790-70a1d5add03f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:22.877373 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-66837a33-a977-4276-9790-70a1d5add03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:22.905642 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0887c-f431-4a18-b4d1-0bb2cca4d8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:22.906528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0887c-f431-4a18-b4d1-0bb2cca4d8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:22.906711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0887c-f431-4a18-b4d1-0bb2cca4d8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:22.907573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0887c-f431-4a18-b4d1-0bb2cca4d8b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:22.907896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0887c-f431-4a18-b4d1-0bb2cca4d8b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:22.910909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0887c-f431-4a18-b4d1-0bb2cca4d8b7 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:22.944765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:22.949685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1264/2533] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:22 2022] POST /identity/v3/users => generated 284 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:22.957470 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0887c-f431-4a18-b4d1-0bb2cca4d8b7 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:22.969763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0887c-f431-4a18-b4d1-0bb2cca4d8b7 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:22.978219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a6f9c-aeb8-463f-9eeb-2e25857f400c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:22.978957 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a6f9c-aeb8-463f-9eeb-2e25857f400c admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:22.979242 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a6f9c-aeb8-463f-9eeb-2e25857f400c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:22.979553 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a6f9c-aeb8-463f-9eeb-2e25857f400c admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:22.980310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a6f9c-aeb8-463f-9eeb-2e25857f400c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:22.982104 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a6f9c-aeb8-463f-9eeb-2e25857f400c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:22.986339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66837a33-a977-4276-9790-70a1d5add03f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:22.995742 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1265/2534] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:22.995979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:22.997658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:22.998203 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:22.998761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] PATH_INFO: `/v3/users/740b427344a741eeb00b764b36ddc223` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:23.000431 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a6f9c-aeb8-463f-9eeb-2e25857f400c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:23.002467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=740b427344a741eeb00b764b36ddc223)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:23.006657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:23.015233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a6f9c-aeb8-463f-9eeb-2e25857f400c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:23.022069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88baea57-591b-4175-bb00-ea881c708853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:23.023045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baea57-591b-4175-bb00-ea881c708853 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:23.023517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baea57-591b-4175-bb00-ea881c708853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:23.023872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baea57-591b-4175-bb00-ea881c708853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:23.024708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88baea57-591b-4175-bb00-ea881c708853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:23.026645 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88baea57-591b-4175-bb00-ea881c708853 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:23.039864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1266/2535] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:23.044567 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88baea57-591b-4175-bb00-ea881c708853 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:23.056798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88baea57-591b-4175-bb00-ea881c708853 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:23.062916 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2006065e-4033-4783-8ad9-d727268f2161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:23.064205 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006065e-4033-4783-8ad9-d727268f2161 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:23.064506 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006065e-4033-4783-8ad9-d727268f2161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:23.064786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2006065e-4033-4783-8ad9-d727268f2161 admin admin] PATH_INFO: `/v3/projects/0ba44434db304af8aa73fc03e4b4396e/users/bf162172d3be4873aa4d78582a5fb2f6/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:23.088985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006065e-4033-4783-8ad9-d727268f2161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba44434db304af8aa73fc03e4b4396e, user_id=bf162172d3be4873aa4d78582a5fb2f6, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:23.095744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2006065e-4033-4783-8ad9-d727268f2161 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:23.107513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '740b427344a741eeb00b764b36ddc223'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.109250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '740b427344a741eeb00b764b36ddc223'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.126774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1267/2536] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:23 2022] PUT /identity/v3/projects/0ba44434db304af8aa73fc03e4b4396e/users/bf162172d3be4873aa4d78582a5fb2f6/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:23.129329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '740b427344a741eeb00b764b36ddc223'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.140360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f9840-d042-4e0f-8be0-b1c96abf2736 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:23.140746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f9840-d042-4e0f-8be0-b1c96abf2736 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:23.141031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f9840-d042-4e0f-8be0-b1c96abf2736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:23.144577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '740b427344a741eeb00b764b36ddc223'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.145319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '740b427344a741eeb00b764b36ddc223'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.181927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1270/2537] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:22 2022] DELETE /identity/v3/users/740b427344a741eeb00b764b36ddc223 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:23.197048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-129f9840-d042-4e0f-8be0-b1c96abf2736 None None] MFA Rules not processed for user `bf162172d3be4873aa4d78582a5fb2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:23.198518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-129f9840-d042-4e0f-8be0-b1c96abf2736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:23.291366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:23.298685 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1268/2538] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:23.303560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:23.305741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:23.389073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300168b6-156b-444e-91fa-cc1a121bbc5c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:23.394142 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61eefe5-37aa-4925-8248-e6ae1a8d06e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:23.395002 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61eefe5-37aa-4925-8248-e6ae1a8d06e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:23.395291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61eefe5-37aa-4925-8248-e6ae1a8d06e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:23.395610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61eefe5-37aa-4925-8248-e6ae1a8d06e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:23.396340 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61eefe5-37aa-4925-8248-e6ae1a8d06e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:23.398330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61eefe5-37aa-4925-8248-e6ae1a8d06e8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:23.429113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f9840-d042-4e0f-8be0-b1c96abf2736 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:23.431325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-129f9840-d042-4e0f-8be0-b1c96abf2736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:23.456771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1271/2539] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:23 2022] POST /identity/v3/projects => generated 369 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:23.462923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61eefe5-37aa-4925-8248-e6ae1a8d06e8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:23.476566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f9840-d042-4e0f-8be0-b1c96abf2736 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:23.551414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:23.552587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:23.553228 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:23.553585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] PATH_INFO: `/v3/projects/cbe26d73033a44f98d3d638d584aa139` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:23.555739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbe26d73033a44f98d3d638d584aa139)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:23.557815 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:23.560454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:23.590346 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61eefe5-37aa-4925-8248-e6ae1a8d06e8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:23.596555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48861dd-7b8b-418d-9a14-cbaf5e0b52b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:23.597277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48861dd-7b8b-418d-9a14-cbaf5e0b52b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:23.597565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48861dd-7b8b-418d-9a14-cbaf5e0b52b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:23.597845 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48861dd-7b8b-418d-9a14-cbaf5e0b52b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:23.598587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48861dd-7b8b-418d-9a14-cbaf5e0b52b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:23.601770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48861dd-7b8b-418d-9a14-cbaf5e0b52b6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:23.625550 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbe26d73033a44f98d3d638d584aa139', 'request_id': 'req-cace1a14-08a3-4080-8460-5428c080dce8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.633570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbe26d73033a44f98d3d638d584aa139', 'request_id': 'req-cace1a14-08a3-4080-8460-5428c080dce8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.641121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbe26d73033a44f98d3d638d584aa139', 'request_id': 'req-cace1a14-08a3-4080-8460-5428c080dce8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.641817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1272/2540] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:23 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:23.648581 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48861dd-7b8b-418d-9a14-cbaf5e0b52b6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:23.661149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] The token cache is being invalidate because project cbe26d73033a44f98d3d638d584aa139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:23.661743 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.662866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.665716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1269/2541] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:23 2022] DELETE /identity/v3/projects/cbe26d73033a44f98d3d638d584aa139 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:23.667056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48861dd-7b8b-418d-9a14-cbaf5e0b52b6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:23.671424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:23.673150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:23.731635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af137e39-51de-45ac-ba7e-654ed4715855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:23.732309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af137e39-51de-45ac-ba7e-654ed4715855 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:23.732586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af137e39-51de-45ac-ba7e-654ed4715855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:23.732939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af137e39-51de-45ac-ba7e-654ed4715855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:23.733611 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af137e39-51de-45ac-ba7e-654ed4715855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:23.735242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af137e39-51de-45ac-ba7e-654ed4715855 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:23.749071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace1a14-08a3-4080-8460-5428c080dce8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:23.750773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1273/2542] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:23.755296 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af137e39-51de-45ac-ba7e-654ed4715855 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:23.757307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-af137e39-51de-45ac-ba7e-654ed4715855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:23.757526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:23.758617 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:23.758983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:23.759264 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] PATH_INFO: `/v3/users/47da623af8ea4eb5bed4aad9e730a0a4` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:23.760719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47da623af8ea4eb5bed4aad9e730a0a4)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:23.762683 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:23.773744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af137e39-51de-45ac-ba7e-654ed4715855 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:23.780130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5784717-c190-4122-8a7b-32f41e96485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:23.781327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5784717-c190-4122-8a7b-32f41e96485c admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:23.781656 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5784717-c190-4122-8a7b-32f41e96485c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:23.782026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5784717-c190-4122-8a7b-32f41e96485c admin admin] PATH_INFO: `/v3/projects/0630b032cd45418c9e6cdfb004ba0050/users/64d7f012cc654d1e9652771778e55662/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:23.802507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:23.803352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5784717-c190-4122-8a7b-32f41e96485c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0630b032cd45418c9e6cdfb004ba0050, user_id=64d7f012cc654d1e9652771778e55662, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:23.806750 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5784717-c190-4122-8a7b-32f41e96485c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:23.822492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1274/2543] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:23 2022] PUT /identity/v3/projects/0630b032cd45418c9e6cdfb004ba0050/users/64d7f012cc654d1e9652771778e55662/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:23.823526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47da623af8ea4eb5bed4aad9e730a0a4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.827248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5784717-c190-4122-8a7b-32f41e96485c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:23.833431 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47da623af8ea4eb5bed4aad9e730a0a4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.834551 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47da623af8ea4eb5bed4aad9e730a0a4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.844958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47da623af8ea4eb5bed4aad9e730a0a4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.845851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47da623af8ea4eb5bed4aad9e730a0a4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:23.870357 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1270/2544] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:23 2022] DELETE /identity/v3/users/47da623af8ea4eb5bed4aad9e730a0a4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:23.912463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5784717-c190-4122-8a7b-32f41e96485c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:23.977036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:23.995186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53b200-8e4f-44dd-9b4d-109ab4dd48cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:23.996508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53b200-8e4f-44dd-9b4d-109ab4dd48cd admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:23.997257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53b200-8e4f-44dd-9b4d-109ab4dd48cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:23.997761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53b200-8e4f-44dd-9b4d-109ab4dd48cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:23.999356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53b200-8e4f-44dd-9b4d-109ab4dd48cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:24.003284 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53b200-8e4f-44dd-9b4d-109ab4dd48cd admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:24.019262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1275/2545] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:24.024141 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:24.025814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:24.040312 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541faeb-b144-4e6e-a1cd-3763edb9f748 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:24.047779 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac0308-3f61-472a-983f-59519fe1a9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:24.048866 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac0308-3f61-472a-983f-59519fe1a9d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.049821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac0308-3f61-472a-983f-59519fe1a9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.050227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac0308-3f61-472a-983f-59519fe1a9d5 admin admin] PATH_INFO: `/v3/projects/0630b032cd45418c9e6cdfb004ba0050/users/64d7f012cc654d1e9652771778e55662/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:24.052637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac0308-3f61-472a-983f-59519fe1a9d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0630b032cd45418c9e6cdfb004ba0050, user_id=64d7f012cc654d1e9652771778e55662, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:24.055722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac0308-3f61-472a-983f-59519fe1a9d5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:24.068191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1271/2546] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:24 2022] PUT /identity/v3/projects/0630b032cd45418c9e6cdfb004ba0050/users/64d7f012cc654d1e9652771778e55662/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:24.078619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e0038-c0ce-4544-a712-1c90fb9937d9 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.078906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e0038-c0ce-4544-a712-1c90fb9937d9 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.079187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e0038-c0ce-4544-a712-1c90fb9937d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:24.105411 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac0308-3f61-472a-983f-59519fe1a9d5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:24.107336 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6dac0308-3f61-472a-983f-59519fe1a9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:24.118942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-353e0038-c0ce-4544-a712-1c90fb9937d9 None None] MFA Rules not processed for user `64d7f012cc654d1e9652771778e55662`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:24.121312 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-353e0038-c0ce-4544-a712-1c90fb9937d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:24.209846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac0308-3f61-472a-983f-59519fe1a9d5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:24.214946 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40568191-930d-4af7-9ce2-363927b74162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:24.215873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40568191-930d-4af7-9ce2-363927b74162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.216159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40568191-930d-4af7-9ce2-363927b74162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.216442 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40568191-930d-4af7-9ce2-363927b74162 admin admin] PATH_INFO: `/v3/projects/b31c6f6aabc546d5b178e799c788200d` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:24.217801 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40568191-930d-4af7-9ce2-363927b74162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b31c6f6aabc546d5b178e799c788200d)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:24.220199 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40568191-930d-4af7-9ce2-363927b74162 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:24.259553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1276/2547] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:24.272821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e0038-c0ce-4544-a712-1c90fb9937d9 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:24.275834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-353e0038-c0ce-4544-a712-1c90fb9937d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:24.284968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-40568191-930d-4af7-9ce2-363927b74162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31c6f6aabc546d5b178e799c788200d', 'request_id': 'req-40568191-930d-4af7-9ce2-363927b74162'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:24.294707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-40568191-930d-4af7-9ce2-363927b74162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31c6f6aabc546d5b178e799c788200d', 'request_id': 'req-40568191-930d-4af7-9ce2-363927b74162'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:24.304481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-40568191-930d-4af7-9ce2-363927b74162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31c6f6aabc546d5b178e799c788200d', 'request_id': 'req-40568191-930d-4af7-9ce2-363927b74162'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:24.322607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-40568191-930d-4af7-9ce2-363927b74162 admin admin] The token cache is being invalidate because project b31c6f6aabc546d5b178e799c788200d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:24.323253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-40568191-930d-4af7-9ce2-363927b74162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:24.324204 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-40568191-930d-4af7-9ce2-363927b74162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:24.326576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1272/2548] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:24 2022] DELETE /identity/v3/projects/b31c6f6aabc546d5b178e799c788200d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:24.334152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5acc13-a924-44fe-bc59-6058942bb19f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.334424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5acc13-a924-44fe-bc59-6058942bb19f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.334718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5acc13-a924-44fe-bc59-6058942bb19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:24.359268 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-fa5acc13-a924-44fe-bc59-6058942bb19f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:24.361457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fa5acc13-a924-44fe-bc59-6058942bb19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:24.383946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e0038-c0ce-4544-a712-1c90fb9937d9 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:24.389823 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:24.463504 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1273/2549] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:24.465549 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf41ea-75e3-4e83-8abc-0b6205fa8e50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:24.466631 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf41ea-75e3-4e83-8abc-0b6205fa8e50 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.467128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf41ea-75e3-4e83-8abc-0b6205fa8e50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.467634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf41ea-75e3-4e83-8abc-0b6205fa8e50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:24.470276 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531c010-e4bf-4390-ad09-1ca264775b32 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.470572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531c010-e4bf-4390-ad09-1ca264775b32 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.470822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-aebf41ea-75e3-4e83-8abc-0b6205fa8e50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:24.470912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531c010-e4bf-4390-ad09-1ca264775b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:24.494213 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a531c010-e4bf-4390-ad09-1ca264775b32 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:24.495723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a531c010-e4bf-4390-ad09-1ca264775b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:24.507632 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1274/2550] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:24.513162 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531c010-e4bf-4390-ad09-1ca264775b32 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:24.529700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531c010-e4bf-4390-ad09-1ca264775b32 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:24.536520 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15981d1e-f0cc-4011-aef8-2da1b5b23a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:24.537218 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15981d1e-f0cc-4011-aef8-2da1b5b23a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.537508 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15981d1e-f0cc-4011-aef8-2da1b5b23a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.537787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15981d1e-f0cc-4011-aef8-2da1b5b23a79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:24.539281 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15981d1e-f0cc-4011-aef8-2da1b5b23a79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:24.539638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15981d1e-f0cc-4011-aef8-2da1b5b23a79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:24.541785 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15981d1e-f0cc-4011-aef8-2da1b5b23a79 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:24.551058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:24.559510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1275/2551] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:18:24.563861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15981d1e-f0cc-4011-aef8-2da1b5b23a79 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:24.577083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15981d1e-f0cc-4011-aef8-2da1b5b23a79 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:24.583075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e4f50-5314-45c3-9dbc-1e666aa1f134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:24.583819 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e4f50-5314-45c3-9dbc-1e666aa1f134 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.584116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e4f50-5314-45c3-9dbc-1e666aa1f134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.584398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e4f50-5314-45c3-9dbc-1e666aa1f134 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:24.585152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e4f50-5314-45c3-9dbc-1e666aa1f134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:24.587135 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e4f50-5314-45c3-9dbc-1e666aa1f134 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:24.612251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf41ea-75e3-4e83-8abc-0b6205fa8e50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:24.613641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf41ea-75e3-4e83-8abc-0b6205fa8e50 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:24.630070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1276/2552] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:24 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:24.633595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e4f50-5314-45c3-9dbc-1e666aa1f134 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:24.653170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1277/2553] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:24 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:18:24.772324 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e4f50-5314-45c3-9dbc-1e666aa1f134 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:24.778201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6915355-1b3e-45b8-a877-e3474bf8fc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:24.779000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6915355-1b3e-45b8-a877-e3474bf8fc59 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.779294 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6915355-1b3e-45b8-a877-e3474bf8fc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.779610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6915355-1b3e-45b8-a877-e3474bf8fc59 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:24.780361 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6915355-1b3e-45b8-a877-e3474bf8fc59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:24.782358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6915355-1b3e-45b8-a877-e3474bf8fc59 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:24.811970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1277/2554] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:24 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:24.817261 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf41ea-75e3-4e83-8abc-0b6205fa8e50 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:24.828566 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf41ea-75e3-4e83-8abc-0b6205fa8e50 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:24.835911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06394e5c-f015-4b4c-bd2b-8912e069c25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:24.836851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06394e5c-f015-4b4c-bd2b-8912e069c25f admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.837443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06394e5c-f015-4b4c-bd2b-8912e069c25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.837828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06394e5c-f015-4b4c-bd2b-8912e069c25f admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:24.838901 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06394e5c-f015-4b4c-bd2b-8912e069c25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:24.840858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06394e5c-f015-4b4c-bd2b-8912e069c25f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:24.860264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1278/2555] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:24.864884 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6915355-1b3e-45b8-a877-e3474bf8fc59 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:24.875786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6915355-1b3e-45b8-a877-e3474bf8fc59 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:24.881487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb29383-311f-4ef6-ab76-f7028c2d3bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:24.882188 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb29383-311f-4ef6-ab76-f7028c2d3bdb admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.882482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb29383-311f-4ef6-ab76-f7028c2d3bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.882775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb29383-311f-4ef6-ab76-f7028c2d3bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:24.883568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb29383-311f-4ef6-ab76-f7028c2d3bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:24.885247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb29383-311f-4ef6-ab76-f7028c2d3bdb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:24.896480 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1278/2556] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:24.901437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06394e5c-f015-4b4c-bd2b-8912e069c25f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:24.916190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06394e5c-f015-4b4c-bd2b-8912e069c25f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:24.922900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3e3ac-e6d7-400e-932a-910fbf10495e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:24.924645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3e3ac-e6d7-400e-932a-910fbf10495e admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.925309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3e3ac-e6d7-400e-932a-910fbf10495e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.925708 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3e3ac-e6d7-400e-932a-910fbf10495e admin admin] PATH_INFO: `/v3/projects/b1964bde402a401eb1bc5dd6c7d853be/users/7c16bedea63145b88ddcff8c45131ae3/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:24.955867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3e3ac-e6d7-400e-932a-910fbf10495e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1964bde402a401eb1bc5dd6c7d853be, user_id=7c16bedea63145b88ddcff8c45131ae3, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:24.960070 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3e3ac-e6d7-400e-932a-910fbf10495e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:24.978805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1279/2557] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:24 2022] PUT /identity/v3/projects/b1964bde402a401eb1bc5dd6c7d853be/users/7c16bedea63145b88ddcff8c45131ae3/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:24.995487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9b612-40f3-46cc-bb0d-b83898b71d36 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:24.997705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9b612-40f3-46cc-bb0d-b83898b71d36 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:24.997705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9b612-40f3-46cc-bb0d-b83898b71d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:25.038432 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a2b9b612-40f3-46cc-bb0d-b83898b71d36 None None] MFA Rules not processed for user `7c16bedea63145b88ddcff8c45131ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:25.039895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a2b9b612-40f3-46cc-bb0d-b83898b71d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:25.116726 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:25.151454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1279/2558] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:25.156091 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3e3ac-e6d7-400e-932a-910fbf10495e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:25.278628 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3e3ac-e6d7-400e-932a-910fbf10495e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:25.286912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3f052-4fff-49f6-8fdc-cb66c691c1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:25.288806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3f052-4fff-49f6-8fdc-cb66c691c1af admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:25.289163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3f052-4fff-49f6-8fdc-cb66c691c1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:25.289718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3f052-4fff-49f6-8fdc-cb66c691c1af admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:25.290610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3f052-4fff-49f6-8fdc-cb66c691c1af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:25.293556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3f052-4fff-49f6-8fdc-cb66c691c1af admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:25.335530 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1280/2559] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:25 2022] POST /identity/v3/projects => generated 363 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:25.337684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9b612-40f3-46cc-bb0d-b83898b71d36 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:25.352940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9b612-40f3-46cc-bb0d-b83898b71d36 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:25.442731 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f4fb4-cf15-4b93-8971-59515dba8140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:25.443590 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f4fb4-cf15-4b93-8971-59515dba8140 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:25.443939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f4fb4-cf15-4b93-8971-59515dba8140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:25.444322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f4fb4-cf15-4b93-8971-59515dba8140 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:25.445354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f4fb4-cf15-4b93-8971-59515dba8140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:25.448012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f4fb4-cf15-4b93-8971-59515dba8140 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:25.478337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1280/2560] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:25 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:25.484081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3f052-4fff-49f6-8fdc-cb66c691c1af admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:25.508380 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3f052-4fff-49f6-8fdc-cb66c691c1af admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:25.514519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670430f-aa44-49fd-bcb0-d97f9a8fe7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:25.515709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670430f-aa44-49fd-bcb0-d97f9a8fe7db admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:25.516083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670430f-aa44-49fd-bcb0-d97f9a8fe7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:25.516381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670430f-aa44-49fd-bcb0-d97f9a8fe7db admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:25.517603 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670430f-aa44-49fd-bcb0-d97f9a8fe7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:25.520234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670430f-aa44-49fd-bcb0-d97f9a8fe7db admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:25.537757 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1281/2561] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:25.543714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f4fb4-cf15-4b93-8971-59515dba8140 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:25.559006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f4fb4-cf15-4b93-8971-59515dba8140 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:25.566580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cffcae-aa31-4ba7-b7cc-8d35a9da4d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:25.568005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cffcae-aa31-4ba7-b7cc-8d35a9da4d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:25.568448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cffcae-aa31-4ba7-b7cc-8d35a9da4d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:25.568877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cffcae-aa31-4ba7-b7cc-8d35a9da4d8e admin admin] PATH_INFO: `/v3/projects/3a46b95f5568488689be997d34553184/users/51b0db4e720c4354ae18c4fe5d671d55/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:25.597229 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cffcae-aa31-4ba7-b7cc-8d35a9da4d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a46b95f5568488689be997d34553184, user_id=51b0db4e720c4354ae18c4fe5d671d55, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:25.602500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cffcae-aa31-4ba7-b7cc-8d35a9da4d8e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:25.626099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1281/2562] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:25 2022] PUT /identity/v3/projects/3a46b95f5568488689be997d34553184/users/51b0db4e720c4354ae18c4fe5d671d55/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:25.632282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670430f-aa44-49fd-bcb0-d97f9a8fe7db admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:25.647066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670430f-aa44-49fd-bcb0-d97f9a8fe7db admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:25.733242 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cf56f-5957-41d9-ade7-7015946edd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:25.733941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cf56f-5957-41d9-ade7-7015946edd13 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:25.734230 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cf56f-5957-41d9-ade7-7015946edd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:25.734595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cf56f-5957-41d9-ade7-7015946edd13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:25.735250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cf56f-5957-41d9-ade7-7015946edd13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:25.737026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cf56f-5957-41d9-ade7-7015946edd13 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:25.764280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1282/2563] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:25.772737 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cffcae-aa31-4ba7-b7cc-8d35a9da4d8e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:25.789294 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cffcae-aa31-4ba7-b7cc-8d35a9da4d8e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:25.798532 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8ea5a-254f-426b-b81a-73585dd669c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:25.799647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8ea5a-254f-426b-b81a-73585dd669c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:25.800088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8ea5a-254f-426b-b81a-73585dd669c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:25.801362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8ea5a-254f-426b-b81a-73585dd669c2 admin admin] PATH_INFO: `/v3/projects/3a46b95f5568488689be997d34553184/users/51b0db4e720c4354ae18c4fe5d671d55/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:25.808071 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8ea5a-254f-426b-b81a-73585dd669c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a46b95f5568488689be997d34553184, user_id=51b0db4e720c4354ae18c4fe5d671d55, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:25.809396 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8ea5a-254f-426b-b81a-73585dd669c2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:25.825560 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1282/2564] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:25 2022] PUT /identity/v3/projects/3a46b95f5568488689be997d34553184/users/51b0db4e720c4354ae18c4fe5d671d55/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:25.847409 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379e177-1762-4759-aaba-32fd64485ed0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:25.847880 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379e177-1762-4759-aaba-32fd64485ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:25.848314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379e177-1762-4759-aaba-32fd64485ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:25.912463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-7379e177-1762-4759-aaba-32fd64485ed0 None None] MFA Rules not processed for user `51b0db4e720c4354ae18c4fe5d671d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:25.913928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7379e177-1762-4759-aaba-32fd64485ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:25.920697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:26.059911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1283/2565] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:25 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:26.071083 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8ea5a-254f-426b-b81a-73585dd669c2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:26.072949 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e7b8ea5a-254f-426b-b81a-73585dd669c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:26.167094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8ea5a-254f-426b-b81a-73585dd669c2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:26.171201 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f17e4-c60d-4292-b84e-0e4d4d66ea3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:26.172253 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f17e4-c60d-4292-b84e-0e4d4d66ea3b service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:26.172548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f17e4-c60d-4292-b84e-0e4d4d66ea3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:26.172911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f17e4-c60d-4292-b84e-0e4d4d66ea3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:26.185217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f17e4-c60d-4292-b84e-0e4d4d66ea3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:26.186676 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f17e4-c60d-4292-b84e-0e4d4d66ea3b service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:26.209799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1283/2566] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:26 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:26.394745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379e177-1762-4759-aaba-32fd64485ed0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:26.396751 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7379e177-1762-4759-aaba-32fd64485ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:26.410100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379e177-1762-4759-aaba-32fd64485ed0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:26.414316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120adbc7-ce38-4454-a5bb-710985aac8e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:26.415204 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120adbc7-ce38-4454-a5bb-710985aac8e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:26.415517 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120adbc7-ce38-4454-a5bb-710985aac8e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:26.415830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120adbc7-ce38-4454-a5bb-710985aac8e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:26.481619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120adbc7-ce38-4454-a5bb-710985aac8e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:26.483831 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120adbc7-ce38-4454-a5bb-710985aac8e2 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:26.497452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1284/2567] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:27.035191 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f17e4-c60d-4292-b84e-0e4d4d66ea3b service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:27.036907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e88f17e4-c60d-4292-b84e-0e4d4d66ea3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:27.092703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:27.112889 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f17e4-c60d-4292-b84e-0e4d4d66ea3b service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:27.118487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:27.119841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:27.120166 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:27.120477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] PATH_INFO: `/v3/users/bf162172d3be4873aa4d78582a5fb2f6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:27.121826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf162172d3be4873aa4d78582a5fb2f6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:27.124084 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:27.181935 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf162172d3be4873aa4d78582a5fb2f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:27.190782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf162172d3be4873aa4d78582a5fb2f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:27.191837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf162172d3be4873aa4d78582a5fb2f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:27.205175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf162172d3be4873aa4d78582a5fb2f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:27.206402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf162172d3be4873aa4d78582a5fb2f6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:27.239894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1284/2568] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:27 2022] DELETE /identity/v3/users/bf162172d3be4873aa4d78582a5fb2f6 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:27.244740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eaebd-9bf9-4e37-a327-72927a883e8f None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:27.245568 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eaebd-9bf9-4e37-a327-72927a883e8f None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:27.246218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eaebd-9bf9-4e37-a327-72927a883e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:27.266738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-369eaebd-9bf9-4e37-a327-72927a883e8f None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:27.268248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-369eaebd-9bf9-4e37-a327-72927a883e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:27.332110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1285/2569] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:27.341814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:27.343259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:27.406113 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b3700-183b-4086-9457-582b0d9531b6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:27.412510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e7a5-445e-4ae4-bc6b-94319b46b2fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:27.413568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e7a5-445e-4ae4-bc6b-94319b46b2fd service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:27.413903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e7a5-445e-4ae4-bc6b-94319b46b2fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:27.414232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e7a5-445e-4ae4-bc6b-94319b46b2fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:27.427749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9e7a5-445e-4ae4-bc6b-94319b46b2fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:27.429508 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9e7a5-445e-4ae4-bc6b-94319b46b2fd service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:27.440642 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1285/2570] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:27.841718 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369eaebd-9bf9-4e37-a327-72927a883e8f None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:27.843986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-369eaebd-9bf9-4e37-a327-72927a883e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:27.868777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369eaebd-9bf9-4e37-a327-72927a883e8f None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:27.882764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:27.884078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:27.887494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:27.888374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] PATH_INFO: `/v3/projects/0ba44434db304af8aa73fc03e4b4396e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:27.891231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba44434db304af8aa73fc03e4b4396e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:27.897755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:27.964827 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:27.982720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba44434db304af8aa73fc03e4b4396e', 'request_id': 'req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:27.995284 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba44434db304af8aa73fc03e4b4396e', 'request_id': 'req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.011668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba44434db304af8aa73fc03e4b4396e', 'request_id': 'req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.046069 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] The token cache is being invalidate because project 0ba44434db304af8aa73fc03e4b4396e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:28.046607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.047802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.051191 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1286/2571] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:27 2022] DELETE /identity/v3/projects/0ba44434db304af8aa73fc03e4b4396e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:28.058373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e7a5-445e-4ae4-bc6b-94319b46b2fd service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:28.071845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e7a5-445e-4ae4-bc6b-94319b46b2fd service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:28.142121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa6ca2-bad9-4293-ac4f-fa3bf6a4c15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:28.143694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa6ca2-bad9-4293-ac4f-fa3bf6a4c15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:28.143979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa6ca2-bad9-4293-ac4f-fa3bf6a4c15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:28.144256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa6ca2-bad9-4293-ac4f-fa3bf6a4c15f admin admin] PATH_INFO: `/v3/users/64d7f012cc654d1e9652771778e55662` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:28.145543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa6ca2-bad9-4293-ac4f-fa3bf6a4c15f admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d7f012cc654d1e9652771778e55662)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:28.147647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa6ca2-bad9-4293-ac4f-fa3bf6a4c15f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:28.160095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:28.222181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-07aa6ca2-bad9-4293-ac4f-fa3bf6a4c15f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d7f012cc654d1e9652771778e55662'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.233009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-07aa6ca2-bad9-4293-ac4f-fa3bf6a4c15f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d7f012cc654d1e9652771778e55662'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.233912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-07aa6ca2-bad9-4293-ac4f-fa3bf6a4c15f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d7f012cc654d1e9652771778e55662'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.245549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-07aa6ca2-bad9-4293-ac4f-fa3bf6a4c15f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d7f012cc654d1e9652771778e55662'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.247265 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-07aa6ca2-bad9-4293-ac4f-fa3bf6a4c15f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d7f012cc654d1e9652771778e55662'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.272664 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1286/2572] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:28 2022] DELETE /identity/v3/users/64d7f012cc654d1e9652771778e55662 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:28.673796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:28.678129 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:28.702867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e74ae-ba85-4de0-8b12-06aa426a3cfa admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:28.787048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:28.788344 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:28.788778 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:28.789260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d admin admin] PATH_INFO: `/v3/projects/0630b032cd45418c9e6cdfb004ba0050` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:28.791233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0630b032cd45418c9e6cdfb004ba0050)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:28.793719 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:28.853305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0630b032cd45418c9e6cdfb004ba0050', 'request_id': 'req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.862174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0630b032cd45418c9e6cdfb004ba0050', 'request_id': 'req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.874363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0630b032cd45418c9e6cdfb004ba0050', 'request_id': 'req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.898284 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d admin admin] The token cache is being invalidate because project 0630b032cd45418c9e6cdfb004ba0050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:28.899048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.900271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1dedf309-9ce9-4f8d-8188-39d74ea1b26d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:28.903347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1287/2573] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:28 2022] DELETE /identity/v3/projects/0630b032cd45418c9e6cdfb004ba0050 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:28.909213 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc001f86-b392-499e-9151-4f5b518ffd82 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:28.909734 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc001f86-b392-499e-9151-4f5b518ffd82 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:28.910214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc001f86-b392-499e-9151-4f5b518ffd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:28.941024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bc001f86-b392-499e-9151-4f5b518ffd82 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:28.943320 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bc001f86-b392-499e-9151-4f5b518ffd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:29.056541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1287/2574] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:29.066271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ed0d-f661-4e48-b4c7-e55a825e6d2a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:29.066747 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ed0d-f661-4e48-b4c7-e55a825e6d2a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:29.067260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ed0d-f661-4e48-b4c7-e55a825e6d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:29.098820 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2595ed0d-f661-4e48-b4c7-e55a825e6d2a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:29.100888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2595ed0d-f661-4e48-b4c7-e55a825e6d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:29.113872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1288/2575] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:29.124971 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc001f86-b392-499e-9151-4f5b518ffd82 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:29.138120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc001f86-b392-499e-9151-4f5b518ffd82 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:29.144448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938a225-b6e8-4605-bf0e-c82c1621ca2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:29.145247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938a225-b6e8-4605-bf0e-c82c1621ca2b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:29.145529 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938a225-b6e8-4605-bf0e-c82c1621ca2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:29.145956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938a225-b6e8-4605-bf0e-c82c1621ca2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:29.146786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938a225-b6e8-4605-bf0e-c82c1621ca2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:29.147499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938a225-b6e8-4605-bf0e-c82c1621ca2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:29.149226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938a225-b6e8-4605-bf0e-c82c1621ca2b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:29.169595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1288/2576] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:18:29.175663 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595ed0d-f661-4e48-b4c7-e55a825e6d2a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:29.197803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595ed0d-f661-4e48-b4c7-e55a825e6d2a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:29.207109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b572e8-f480-41b2-a5b4-ecf5317efe14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:29.208222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b572e8-f480-41b2-a5b4-ecf5317efe14 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:29.208645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b572e8-f480-41b2-a5b4-ecf5317efe14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:29.209057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b572e8-f480-41b2-a5b4-ecf5317efe14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:29.210215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b572e8-f480-41b2-a5b4-ecf5317efe14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:29.213830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b572e8-f480-41b2-a5b4-ecf5317efe14 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:29.275964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1289/2577] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:29 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:29.277697 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938a225-b6e8-4605-bf0e-c82c1621ca2b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:29.301462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938a225-b6e8-4605-bf0e-c82c1621ca2b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:29.410257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f5d6f-b093-40a6-b0af-c2fbac6c0160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:29.411763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f5d6f-b093-40a6-b0af-c2fbac6c0160 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:29.412298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f5d6f-b093-40a6-b0af-c2fbac6c0160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:29.412845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f5d6f-b093-40a6-b0af-c2fbac6c0160 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:29.413932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f5d6f-b093-40a6-b0af-c2fbac6c0160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:29.417744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f5d6f-b093-40a6-b0af-c2fbac6c0160 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:29.456690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1289/2578] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:29 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:29.458283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b572e8-f480-41b2-a5b4-ecf5317efe14 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:29.473055 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b572e8-f480-41b2-a5b4-ecf5317efe14 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:29.479229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cceb7-34da-48eb-ac7c-128d71bf74aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:29.480121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cceb7-34da-48eb-ac7c-128d71bf74aa admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:29.480593 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cceb7-34da-48eb-ac7c-128d71bf74aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:29.481134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cceb7-34da-48eb-ac7c-128d71bf74aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:29.482099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551cceb7-34da-48eb-ac7c-128d71bf74aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:29.484636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551cceb7-34da-48eb-ac7c-128d71bf74aa admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:29.522327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1290/2579] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:29.529744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f5d6f-b093-40a6-b0af-c2fbac6c0160 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:29.542300 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:29.554921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f5d6f-b093-40a6-b0af-c2fbac6c0160 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:29.571572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9df7b7-c1c6-483e-87f2-d5e51ae14854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:29.573970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9df7b7-c1c6-483e-87f2-d5e51ae14854 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:29.574484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9df7b7-c1c6-483e-87f2-d5e51ae14854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:29.575451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9df7b7-c1c6-483e-87f2-d5e51ae14854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:29.577136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9df7b7-c1c6-483e-87f2-d5e51ae14854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:29.579163 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9df7b7-c1c6-483e-87f2-d5e51ae14854 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:29.594302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1290/2580] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:29.601279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cceb7-34da-48eb-ac7c-128d71bf74aa admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:29.614771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cceb7-34da-48eb-ac7c-128d71bf74aa admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:29.620352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091f434-39b4-4ae8-b48c-c3c0dffed89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:29.621408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091f434-39b4-4ae8-b48c-c3c0dffed89f admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:29.621794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091f434-39b4-4ae8-b48c-c3c0dffed89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:29.622211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091f434-39b4-4ae8-b48c-c3c0dffed89f admin admin] PATH_INFO: `/v3/projects/2a46877cfaf146588dc4b0179218c024/users/e511a63e84944e219c0d3f41ab5543c9/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:29.638437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:29.647329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091f434-39b4-4ae8-b48c-c3c0dffed89f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a46877cfaf146588dc4b0179218c024, user_id=e511a63e84944e219c0d3f41ab5543c9, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:29.653693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091f434-39b4-4ae8-b48c-c3c0dffed89f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:29.675189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1291/2581] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:29 2022] PUT /identity/v3/projects/2a46877cfaf146588dc4b0179218c024/users/e511a63e84944e219c0d3f41ab5543c9/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:29.687511 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e72b66-5942-4eac-b01e-a97bc28cfbe9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:29.687985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e72b66-5942-4eac-b01e-a97bc28cfbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:29.688278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e72b66-5942-4eac-b01e-a97bc28cfbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:29.721523 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a6e72b66-5942-4eac-b01e-a97bc28cfbe9 None None] MFA Rules not processed for user `e511a63e84944e219c0d3f41ab5543c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:29.723170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a6e72b66-5942-4eac-b01e-a97bc28cfbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:29.800548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1291/2582] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:29.805789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091f434-39b4-4ae8-b48c-c3c0dffed89f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:29.872455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091f434-39b4-4ae8-b48c-c3c0dffed89f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:29.878016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a8d2e-1064-4269-87ff-4784e8a9ada6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:29.878708 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a8d2e-1064-4269-87ff-4784e8a9ada6 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:29.878996 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a8d2e-1064-4269-87ff-4784e8a9ada6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:29.879267 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a8d2e-1064-4269-87ff-4784e8a9ada6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:29.880010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a8d2e-1064-4269-87ff-4784e8a9ada6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:29.881942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a8d2e-1064-4269-87ff-4784e8a9ada6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:29.909848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1292/2583] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:29 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:29.915607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e72b66-5942-4eac-b01e-a97bc28cfbe9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:29.934024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e72b66-5942-4eac-b01e-a97bc28cfbe9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:30.012476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126a649-e00f-4d85-8ec9-c807ea52c85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:30.013139 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126a649-e00f-4d85-8ec9-c807ea52c85d admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:30.013421 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126a649-e00f-4d85-8ec9-c807ea52c85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:30.013835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126a649-e00f-4d85-8ec9-c807ea52c85d admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:30.014563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126a649-e00f-4d85-8ec9-c807ea52c85d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:30.016630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126a649-e00f-4d85-8ec9-c807ea52c85d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:30.041782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1292/2584] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:29 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:30.046250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a8d2e-1064-4269-87ff-4784e8a9ada6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:30.057699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a8d2e-1064-4269-87ff-4784e8a9ada6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:30.062434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42026fc7-b6e2-40ff-9b28-cef25229e000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:30.063124 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42026fc7-b6e2-40ff-9b28-cef25229e000 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:30.063437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42026fc7-b6e2-40ff-9b28-cef25229e000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:30.063720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42026fc7-b6e2-40ff-9b28-cef25229e000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:30.064456 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42026fc7-b6e2-40ff-9b28-cef25229e000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:30.066080 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42026fc7-b6e2-40ff-9b28-cef25229e000 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:30.078111 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1293/2585] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:30.083047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126a649-e00f-4d85-8ec9-c807ea52c85d admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:30.093950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126a649-e00f-4d85-8ec9-c807ea52c85d admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:30.099862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09075e2-bb22-41a3-8b02-b4874abad11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:30.100882 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09075e2-bb22-41a3-8b02-b4874abad11f admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:30.101208 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09075e2-bb22-41a3-8b02-b4874abad11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:30.101589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09075e2-bb22-41a3-8b02-b4874abad11f admin admin] PATH_INFO: `/v3/projects/8bdbeac5f4cd4e32847d616b7c254ded/users/48557487bfa9488494b1d2dafc321455/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:30.121484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09075e2-bb22-41a3-8b02-b4874abad11f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bdbeac5f4cd4e32847d616b7c254ded, user_id=48557487bfa9488494b1d2dafc321455, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:30.124313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09075e2-bb22-41a3-8b02-b4874abad11f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:30.138929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1293/2586] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:30 2022] PUT /identity/v3/projects/8bdbeac5f4cd4e32847d616b7c254ded/users/48557487bfa9488494b1d2dafc321455/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:30.143896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42026fc7-b6e2-40ff-9b28-cef25229e000 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:30.156616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42026fc7-b6e2-40ff-9b28-cef25229e000 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:30.234428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a006c5b-ca78-466b-9709-af628f474299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:30.235146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a006c5b-ca78-466b-9709-af628f474299 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:30.235474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a006c5b-ca78-466b-9709-af628f474299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:30.235825 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a006c5b-ca78-466b-9709-af628f474299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:30.237355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a006c5b-ca78-466b-9709-af628f474299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:30.239014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a006c5b-ca78-466b-9709-af628f474299 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:30.250211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1294/2587] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:30.255266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09075e2-bb22-41a3-8b02-b4874abad11f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:30.266427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09075e2-bb22-41a3-8b02-b4874abad11f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:30.271959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307d116-7d5a-4694-859f-d9d969701d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:30.273286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307d116-7d5a-4694-859f-d9d969701d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:30.273568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307d116-7d5a-4694-859f-d9d969701d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:30.273840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307d116-7d5a-4694-859f-d9d969701d22 admin admin] PATH_INFO: `/v3/projects/8bdbeac5f4cd4e32847d616b7c254ded/users/48557487bfa9488494b1d2dafc321455/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:30.275924 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307d116-7d5a-4694-859f-d9d969701d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bdbeac5f4cd4e32847d616b7c254ded, user_id=48557487bfa9488494b1d2dafc321455, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:30.278563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307d116-7d5a-4694-859f-d9d969701d22 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:30.290572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1294/2588] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:30 2022] PUT /identity/v3/projects/8bdbeac5f4cd4e32847d616b7c254ded/users/48557487bfa9488494b1d2dafc321455/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:30.301334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576526a2-ca9a-43fa-af50-d6f6a13770d4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:30.301724 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576526a2-ca9a-43fa-af50-d6f6a13770d4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:30.302085 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576526a2-ca9a-43fa-af50-d6f6a13770d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:30.333843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-576526a2-ca9a-43fa-af50-d6f6a13770d4 None None] MFA Rules not processed for user `48557487bfa9488494b1d2dafc321455`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:30.335087 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-576526a2-ca9a-43fa-af50-d6f6a13770d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:30.388731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:30.414060 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1295/2589] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:30.426548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307d116-7d5a-4694-859f-d9d969701d22 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:30.428452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c307d116-7d5a-4694-859f-d9d969701d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:30.472881 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:30.495793 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307d116-7d5a-4694-859f-d9d969701d22 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:30.499658 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c987a-9b16-4455-a6a1-b2cb59c4a6e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:30.500325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c987a-9b16-4455-a6a1-b2cb59c4a6e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:30.500608 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c987a-9b16-4455-a6a1-b2cb59c4a6e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:30.500886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c987a-9b16-4455-a6a1-b2cb59c4a6e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:30.512125 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c987a-9b16-4455-a6a1-b2cb59c4a6e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:30.513581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c987a-9b16-4455-a6a1-b2cb59c4a6e6 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:30.525423 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1295/2590] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:30.554803 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576526a2-ca9a-43fa-af50-d6f6a13770d4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:30.631673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576526a2-ca9a-43fa-af50-d6f6a13770d4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:30.636571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:30.637555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:30.637915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:30.638382 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] PATH_INFO: `/v3/users/e511a63e84944e219c0d3f41ab5543c9` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:30.639972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e511a63e84944e219c0d3f41ab5543c9)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:30.641967 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:30.710893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e511a63e84944e219c0d3f41ab5543c9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:30.714042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e511a63e84944e219c0d3f41ab5543c9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:30.728304 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e511a63e84944e219c0d3f41ab5543c9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:30.744264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e511a63e84944e219c0d3f41ab5543c9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:30.744790 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e511a63e84944e219c0d3f41ab5543c9'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:30.772724 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1296/2591] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:30 2022] DELETE /identity/v3/users/e511a63e84944e219c0d3f41ab5543c9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:30.778033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bd521-1e56-4e3a-9280-8348c36d87b4 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:30.778325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bd521-1e56-4e3a-9280-8348c36d87b4 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:30.778479 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bd521-1e56-4e3a-9280-8348c36d87b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:30.800601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4c3bd521-1e56-4e3a-9280-8348c36d87b4 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:30.802931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4c3bd521-1e56-4e3a-9280-8348c36d87b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:30.933886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1296/2592] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:30.951020 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:30.954033 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:31.066119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacbecd-4174-44f9-9c88-f11f82eb0049 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:31.081737 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aebe2d-3781-4dad-bb8e-670c8a924d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:31.083167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aebe2d-3781-4dad-bb8e-670c8a924d23 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:31.084083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aebe2d-3781-4dad-bb8e-670c8a924d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:31.084711 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aebe2d-3781-4dad-bb8e-670c8a924d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:31.113732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aebe2d-3781-4dad-bb8e-670c8a924d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:31.116858 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aebe2d-3781-4dad-bb8e-670c8a924d23 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:31.146070 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1297/2593] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:31.507146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bd521-1e56-4e3a-9280-8348c36d87b4 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:31.509654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4c3bd521-1e56-4e3a-9280-8348c36d87b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:31.529323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bd521-1e56-4e3a-9280-8348c36d87b4 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:31.537334 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:31.538229 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:31.538845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:31.539136 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] PATH_INFO: `/v3/projects/2a46877cfaf146588dc4b0179218c024` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:31.540499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a46877cfaf146588dc4b0179218c024)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:31.542458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:31.612192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a46877cfaf146588dc4b0179218c024', 'request_id': 'req-803f83d4-b90c-49c4-af55-84019eb3ccbf'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:31.615132 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a46877cfaf146588dc4b0179218c024', 'request_id': 'req-803f83d4-b90c-49c4-af55-84019eb3ccbf'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:31.622973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a46877cfaf146588dc4b0179218c024', 'request_id': 'req-803f83d4-b90c-49c4-af55-84019eb3ccbf'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:31.640879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] The token cache is being invalidate because project 2a46877cfaf146588dc4b0179218c024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:31.641171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:31.641912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:31.644015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1297/2594] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:31 2022] DELETE /identity/v3/projects/2a46877cfaf146588dc4b0179218c024 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:31.648199 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aebe2d-3781-4dad-bb8e-670c8a924d23 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:31.658019 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aebe2d-3781-4dad-bb8e-670c8a924d23 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:31.662650 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:31.740298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688bc524-cb7b-4879-990b-c184ad31d055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:31.741947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bc524-cb7b-4879-990b-c184ad31d055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:31.742798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bc524-cb7b-4879-990b-c184ad31d055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:31.743113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bc524-cb7b-4879-990b-c184ad31d055 admin admin] PATH_INFO: `/v3/users/48557487bfa9488494b1d2dafc321455` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:31.748537 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688bc524-cb7b-4879-990b-c184ad31d055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48557487bfa9488494b1d2dafc321455)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:31.752078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688bc524-cb7b-4879-990b-c184ad31d055 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:31.815610 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-688bc524-cb7b-4879-990b-c184ad31d055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48557487bfa9488494b1d2dafc321455'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:31.815957 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-688bc524-cb7b-4879-990b-c184ad31d055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48557487bfa9488494b1d2dafc321455'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:31.826232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-688bc524-cb7b-4879-990b-c184ad31d055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48557487bfa9488494b1d2dafc321455'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:31.840852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-688bc524-cb7b-4879-990b-c184ad31d055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48557487bfa9488494b1d2dafc321455'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:31.841902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-688bc524-cb7b-4879-990b-c184ad31d055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48557487bfa9488494b1d2dafc321455'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:31.873357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1298/2595] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:31 2022] DELETE /identity/v3/users/48557487bfa9488494b1d2dafc321455 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:32.234861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:32.242020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:32.280220 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f83d4-b90c-49c4-af55-84019eb3ccbf admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:32.380364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54903490-b60c-4ee8-a2c5-502bc44388b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:32.381123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54903490-b60c-4ee8-a2c5-502bc44388b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:32.381419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54903490-b60c-4ee8-a2c5-502bc44388b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:32.381814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54903490-b60c-4ee8-a2c5-502bc44388b3 admin admin] PATH_INFO: `/v3/projects/8bdbeac5f4cd4e32847d616b7c254ded` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:32.383570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54903490-b60c-4ee8-a2c5-502bc44388b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bdbeac5f4cd4e32847d616b7c254ded)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:32.386016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54903490-b60c-4ee8-a2c5-502bc44388b3 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:32.389437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:32.453026 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-54903490-b60c-4ee8-a2c5-502bc44388b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bdbeac5f4cd4e32847d616b7c254ded', 'request_id': 'req-54903490-b60c-4ee8-a2c5-502bc44388b3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:32.466382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-54903490-b60c-4ee8-a2c5-502bc44388b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bdbeac5f4cd4e32847d616b7c254ded', 'request_id': 'req-54903490-b60c-4ee8-a2c5-502bc44388b3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:32.478692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-54903490-b60c-4ee8-a2c5-502bc44388b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bdbeac5f4cd4e32847d616b7c254ded', 'request_id': 'req-54903490-b60c-4ee8-a2c5-502bc44388b3'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:32.501439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-54903490-b60c-4ee8-a2c5-502bc44388b3 admin admin] The token cache is being invalidate because project 8bdbeac5f4cd4e32847d616b7c254ded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:32.501968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-54903490-b60c-4ee8-a2c5-502bc44388b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:32.503845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-54903490-b60c-4ee8-a2c5-502bc44388b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:32.506565 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1298/2596] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:32 2022] DELETE /identity/v3/projects/8bdbeac5f4cd4e32847d616b7c254ded => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:32.513986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeb788-d958-4aa5-bf12-a2793b33a46a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:32.514540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeb788-d958-4aa5-bf12-a2793b33a46a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:32.514884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeb788-d958-4aa5-bf12-a2793b33a46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:32.538947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f1eeb788-d958-4aa5-bf12-a2793b33a46a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:32.540643 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f1eeb788-d958-4aa5-bf12-a2793b33a46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:32.630103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1299/2597] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:32.642923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c864e97-766f-475e-9ca7-63372c56c6df None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:32.643773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c864e97-766f-475e-9ca7-63372c56c6df None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:32.643773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c864e97-766f-475e-9ca7-63372c56c6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:32.665419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2c864e97-766f-475e-9ca7-63372c56c6df None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:32.667733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2c864e97-766f-475e-9ca7-63372c56c6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:32.687002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1299/2598] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:32.688834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eeb788-d958-4aa5-bf12-a2793b33a46a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:32.701910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eeb788-d958-4aa5-bf12-a2793b33a46a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:32.709878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9440250-2aca-41e8-83dd-13f9b2bd81bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:32.711063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9440250-2aca-41e8-83dd-13f9b2bd81bc admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:32.711629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9440250-2aca-41e8-83dd-13f9b2bd81bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:32.712135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9440250-2aca-41e8-83dd-13f9b2bd81bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:32.713585 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9440250-2aca-41e8-83dd-13f9b2bd81bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:32.714161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9440250-2aca-41e8-83dd-13f9b2bd81bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:32.718146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9440250-2aca-41e8-83dd-13f9b2bd81bc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:32.743606 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1300/2599] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:18:32.748975 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c864e97-766f-475e-9ca7-63372c56c6df None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:32.764308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c864e97-766f-475e-9ca7-63372c56c6df None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:32.771042 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6584c-927d-4740-87e6-3fef79c491f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:32.772209 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6584c-927d-4740-87e6-3fef79c491f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:32.772707 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6584c-927d-4740-87e6-3fef79c491f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:32.773196 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6584c-927d-4740-87e6-3fef79c491f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:32.774494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6584c-927d-4740-87e6-3fef79c491f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:32.778375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6584c-927d-4740-87e6-3fef79c491f8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:32.821963 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1300/2600] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:32 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:32.827991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9440250-2aca-41e8-83dd-13f9b2bd81bc admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:32.840135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9440250-2aca-41e8-83dd-13f9b2bd81bc admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:32.851946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:32.927990 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c723c-ba6d-446f-9d95-e4bbddb5936c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:32.929453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c723c-ba6d-446f-9d95-e4bbddb5936c admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:32.929975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c723c-ba6d-446f-9d95-e4bbddb5936c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:32.930495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c723c-ba6d-446f-9d95-e4bbddb5936c admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:32.931883 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c723c-ba6d-446f-9d95-e4bbddb5936c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:32.935900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c723c-ba6d-446f-9d95-e4bbddb5936c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:32.970739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1301/2601] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:32 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:32.976710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6584c-927d-4740-87e6-3fef79c491f8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:32.989368 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6584c-927d-4740-87e6-3fef79c491f8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:32.994202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb0661-b6df-468f-9a2c-46aad64c01f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:32.994890 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb0661-b6df-468f-9a2c-46aad64c01f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:32.995180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb0661-b6df-468f-9a2c-46aad64c01f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:32.995499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb0661-b6df-468f-9a2c-46aad64c01f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:32.996241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb0661-b6df-468f-9a2c-46aad64c01f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:32.998149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb0661-b6df-468f-9a2c-46aad64c01f6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:33.012908 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1301/2602] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:33.017638 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c723c-ba6d-446f-9d95-e4bbddb5936c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:33.032763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c723c-ba6d-446f-9d95-e4bbddb5936c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:33.041032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a37b1-9661-463b-b0e4-1d78e98b35c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:33.041982 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a37b1-9661-463b-b0e4-1d78e98b35c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:33.042523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a37b1-9661-463b-b0e4-1d78e98b35c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:33.043001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a37b1-9661-463b-b0e4-1d78e98b35c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:33.044415 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a37b1-9661-463b-b0e4-1d78e98b35c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:33.047612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a37b1-9661-463b-b0e4-1d78e98b35c9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:33.069083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1302/2603] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:33.074050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb0661-b6df-468f-9a2c-46aad64c01f6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:33.092341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb0661-b6df-468f-9a2c-46aad64c01f6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:33.101463 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca876a-fb50-47e4-9445-852da9684be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:33.103218 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca876a-fb50-47e4-9445-852da9684be8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:33.103661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca876a-fb50-47e4-9445-852da9684be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:33.104169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca876a-fb50-47e4-9445-852da9684be8 admin admin] PATH_INFO: `/v3/projects/94432f6569274238a86b9695977c30db/users/49c303616d64446dba3a9fab217a051e/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:33.130648 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca876a-fb50-47e4-9445-852da9684be8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94432f6569274238a86b9695977c30db, user_id=49c303616d64446dba3a9fab217a051e, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:33.133493 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca876a-fb50-47e4-9445-852da9684be8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:33.154299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1302/2604] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:33 2022] PUT /identity/v3/projects/94432f6569274238a86b9695977c30db/users/49c303616d64446dba3a9fab217a051e/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:33.163673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e56087-6faf-4388-9e56-f3d02364d60d None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:33.164195 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e56087-6faf-4388-9e56-f3d02364d60d None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:33.164721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e56087-6faf-4388-9e56-f3d02364d60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:33.199319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e7e56087-6faf-4388-9e56-f3d02364d60d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:33.199319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e7e56087-6faf-4388-9e56-f3d02364d60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:33.284127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1303/2605] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:33.295259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca876a-fb50-47e4-9445-852da9684be8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:33.297231 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-02ca876a-fb50-47e4-9445-852da9684be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:33.331777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e56087-6faf-4388-9e56-f3d02364d60d None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:33.333292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e7e56087-6faf-4388-9e56-f3d02364d60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:33.346702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e56087-6faf-4388-9e56-f3d02364d60d None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:33.352567 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:33.353814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:33.354249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:33.354629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] PATH_INFO: `/v3/users/7c16bedea63145b88ddcff8c45131ae3` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:33.356513 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c16bedea63145b88ddcff8c45131ae3)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:33.358668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:33.391100 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca876a-fb50-47e4-9445-852da9684be8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:33.397024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0142b8c-3053-43bf-9f2a-c19487ce6806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:33.397941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0142b8c-3053-43bf-9f2a-c19487ce6806 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:33.398337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0142b8c-3053-43bf-9f2a-c19487ce6806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:33.398930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0142b8c-3053-43bf-9f2a-c19487ce6806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:33.413117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0142b8c-3053-43bf-9f2a-c19487ce6806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:33.415033 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0142b8c-3053-43bf-9f2a-c19487ce6806 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:33.428488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1303/2606] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:18:33.428944 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c16bedea63145b88ddcff8c45131ae3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:33.429947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c16bedea63145b88ddcff8c45131ae3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:33.438441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c16bedea63145b88ddcff8c45131ae3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:33.448023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c16bedea63145b88ddcff8c45131ae3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:33.448824 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c16bedea63145b88ddcff8c45131ae3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:33.472345 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1304/2607] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:33 2022] DELETE /identity/v3/users/7c16bedea63145b88ddcff8c45131ae3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:33.478024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56afd02-86f4-41b9-a908-c7121411353a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:33.478513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56afd02-86f4-41b9-a908-c7121411353a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:33.478856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56afd02-86f4-41b9-a908-c7121411353a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:33.511595 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e56afd02-86f4-41b9-a908-c7121411353a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:33.513807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e56afd02-86f4-41b9-a908-c7121411353a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:33.570827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:33.644460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1304/2608] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:33.664935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:33.667564 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:33.677667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:33.776522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df148285-38ff-4671-b5e9-3e2e15a40eac admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:33.780522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e0bcf-6c0d-4461-9f42-fcf43699d1ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:33.781218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e0bcf-6c0d-4461-9f42-fcf43699d1ff service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:33.781614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e0bcf-6c0d-4461-9f42-fcf43699d1ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:33.781826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e0bcf-6c0d-4461-9f42-fcf43699d1ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:33.792869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e0bcf-6c0d-4461-9f42-fcf43699d1ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:33.794433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e0bcf-6c0d-4461-9f42-fcf43699d1ff service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:33.807464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1305/2609] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:34.366226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ccdc3-5299-44a7-b2c4-3ab4aabdf19d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:34.366649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ccdc3-5299-44a7-b2c4-3ab4aabdf19d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:34.367139 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ccdc3-5299-44a7-b2c4-3ab4aabdf19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:34.399934 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-665ccdc3-5299-44a7-b2c4-3ab4aabdf19d None None] MFA Rules not processed for user `49c303616d64446dba3a9fab217a051e`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:34.401510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-665ccdc3-5299-44a7-b2c4-3ab4aabdf19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:34.479010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e0bcf-6c0d-4461-9f42-fcf43699d1ff service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:34.480667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c30e0bcf-6c0d-4461-9f42-fcf43699d1ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:34.503768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e0bcf-6c0d-4461-9f42-fcf43699d1ff service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:34.504950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1305/2610] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:34.509370 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:34.509630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ccdc3-5299-44a7-b2c4-3ab4aabdf19d None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:34.510282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:34.510546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:34.510838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] PATH_INFO: `/v3/projects/b1964bde402a401eb1bc5dd6c7d853be` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:34.511577 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-665ccdc3-5299-44a7-b2c4-3ab4aabdf19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:34.512145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1964bde402a401eb1bc5dd6c7d853be)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:34.514188 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:34.530142 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ccdc3-5299-44a7-b2c4-3ab4aabdf19d None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:34.536973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf69cd6-dd29-4ea1-bb3c-eeba4eb60d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:34.537673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf69cd6-dd29-4ea1-bb3c-eeba4eb60d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:34.538097 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf69cd6-dd29-4ea1-bb3c-eeba4eb60d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:34.538382 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf69cd6-dd29-4ea1-bb3c-eeba4eb60d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:34.539277 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf69cd6-dd29-4ea1-bb3c-eeba4eb60d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:34.542491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf69cd6-dd29-4ea1-bb3c-eeba4eb60d8b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:34.583159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1306/2611] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:34 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:34.587485 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf69cd6-dd29-4ea1-bb3c-eeba4eb60d8b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:34.591951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1964bde402a401eb1bc5dd6c7d853be', 'request_id': 'req-853b6be7-4789-4bf8-a38b-81bf38ee2b92'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.608826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1964bde402a401eb1bc5dd6c7d853be', 'request_id': 'req-853b6be7-4789-4bf8-a38b-81bf38ee2b92'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.620648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1964bde402a401eb1bc5dd6c7d853be', 'request_id': 'req-853b6be7-4789-4bf8-a38b-81bf38ee2b92'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.643654 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] The token cache is being invalidate because project b1964bde402a401eb1bc5dd6c7d853be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:34.644278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.645473 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.648307 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1306/2612] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:34 2022] DELETE /identity/v3/projects/b1964bde402a401eb1bc5dd6c7d853be => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:34.653003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:34.655254 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:34.663693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf69cd6-dd29-4ea1-bb3c-eeba4eb60d8b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:34.670010 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e620f0-20fd-46a1-b871-2e659dbd167a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:34.670790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e620f0-20fd-46a1-b871-2e659dbd167a admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:34.671149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e620f0-20fd-46a1-b871-2e659dbd167a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:34.671535 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e620f0-20fd-46a1-b871-2e659dbd167a admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:34.672254 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e620f0-20fd-46a1-b871-2e659dbd167a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:34.674795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e620f0-20fd-46a1-b871-2e659dbd167a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:34.697032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1307/2613] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:34 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:34.701202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e620f0-20fd-46a1-b871-2e659dbd167a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:34.702990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-64e620f0-20fd-46a1-b871-2e659dbd167a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:34.716987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e620f0-20fd-46a1-b871-2e659dbd167a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:34.723921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574a7dd-1dc8-413f-82cf-bb78d406aadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:34.724616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574a7dd-1dc8-413f-82cf-bb78d406aadd admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:34.724898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574a7dd-1dc8-413f-82cf-bb78d406aadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:34.725386 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574a7dd-1dc8-413f-82cf-bb78d406aadd admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:34.725938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574a7dd-1dc8-413f-82cf-bb78d406aadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:34.728293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574a7dd-1dc8-413f-82cf-bb78d406aadd admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:34.746569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1308/2614] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:34.751620 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574a7dd-1dc8-413f-82cf-bb78d406aadd admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:34.765259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574a7dd-1dc8-413f-82cf-bb78d406aadd admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:34.772962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c12a3d-eab4-4898-bd58-6ad77ff303a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:34.774447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c12a3d-eab4-4898-bd58-6ad77ff303a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:34.774939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c12a3d-eab4-4898-bd58-6ad77ff303a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:34.775439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c12a3d-eab4-4898-bd58-6ad77ff303a5 admin admin] PATH_INFO: `/v3/projects/b02ce14ecf394a15afed2dbd3616600e/users/d5f1382ca00c491092affa569b0b9a3f/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:34.780915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b6be7-4789-4bf8-a38b-81bf38ee2b92 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:34.787153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:34.788127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:34.788682 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:34.788964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] PATH_INFO: `/v3/users/51b0db4e720c4354ae18c4fe5d671d55` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:34.790300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51b0db4e720c4354ae18c4fe5d671d55)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:34.792326 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:34.795768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c12a3d-eab4-4898-bd58-6ad77ff303a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b02ce14ecf394a15afed2dbd3616600e, user_id=d5f1382ca00c491092affa569b0b9a3f, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:34.798516 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c12a3d-eab4-4898-bd58-6ad77ff303a5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:34.802373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:34.821070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1309/2615] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:34 2022] PUT /identity/v3/projects/b02ce14ecf394a15afed2dbd3616600e/users/d5f1382ca00c491092affa569b0b9a3f/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:34.826246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c12a3d-eab4-4898-bd58-6ad77ff303a5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:34.845514 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:34.866014 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51b0db4e720c4354ae18c4fe5d671d55'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.867286 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51b0db4e720c4354ae18c4fe5d671d55'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.879255 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51b0db4e720c4354ae18c4fe5d671d55'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.890480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51b0db4e720c4354ae18c4fe5d671d55'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.891629 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51b0db4e720c4354ae18c4fe5d671d55'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:34.905704 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c12a3d-eab4-4898-bd58-6ad77ff303a5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:34.912493 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832da2d4-b5ee-4764-8a0d-4379ae53d7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:34.913284 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832da2d4-b5ee-4764-8a0d-4379ae53d7ab admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:34.913723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832da2d4-b5ee-4764-8a0d-4379ae53d7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:34.914126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832da2d4-b5ee-4764-8a0d-4379ae53d7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:34.915312 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832da2d4-b5ee-4764-8a0d-4379ae53d7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:34.917472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832da2d4-b5ee-4764-8a0d-4379ae53d7ab admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:34.925365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1307/2616] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:34 2022] DELETE /identity/v3/users/51b0db4e720c4354ae18c4fe5d671d55 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:34.936362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1310/2617] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:34.941463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:34.943398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:35.046785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9be375-f4dc-4a92-a747-e911f960c4d3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:35.055349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bcc1d-8452-4eaa-9c5a-d5516de64934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:35.056931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bcc1d-8452-4eaa-9c5a-d5516de64934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:35.057434 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bcc1d-8452-4eaa-9c5a-d5516de64934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:35.057924 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bcc1d-8452-4eaa-9c5a-d5516de64934 admin admin] PATH_INFO: `/v3/projects/b02ce14ecf394a15afed2dbd3616600e/users/d5f1382ca00c491092affa569b0b9a3f/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:35.062126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1bcc1d-8452-4eaa-9c5a-d5516de64934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b02ce14ecf394a15afed2dbd3616600e, user_id=d5f1382ca00c491092affa569b0b9a3f, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:35.067848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1bcc1d-8452-4eaa-9c5a-d5516de64934 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:35.087608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1308/2618] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:34 2022] PUT /identity/v3/projects/b02ce14ecf394a15afed2dbd3616600e/users/d5f1382ca00c491092affa569b0b9a3f/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:35.405408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832da2d4-b5ee-4764-8a0d-4379ae53d7ab admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:35.407881 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-832da2d4-b5ee-4764-8a0d-4379ae53d7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:35.430341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832da2d4-b5ee-4764-8a0d-4379ae53d7ab admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:35.558824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be823c44-fa65-4e2f-b083-7b3c9126bf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:35.559782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be823c44-fa65-4e2f-b083-7b3c9126bf10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:35.560720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be823c44-fa65-4e2f-b083-7b3c9126bf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:35.561032 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be823c44-fa65-4e2f-b083-7b3c9126bf10 admin admin] PATH_INFO: `/v3/projects/3a46b95f5568488689be997d34553184` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:35.562439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be823c44-fa65-4e2f-b083-7b3c9126bf10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a46b95f5568488689be997d34553184)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:35.567151 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be823c44-fa65-4e2f-b083-7b3c9126bf10 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:35.642948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-be823c44-fa65-4e2f-b083-7b3c9126bf10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a46b95f5568488689be997d34553184', 'request_id': 'req-be823c44-fa65-4e2f-b083-7b3c9126bf10'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:35.653037 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-be823c44-fa65-4e2f-b083-7b3c9126bf10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a46b95f5568488689be997d34553184', 'request_id': 'req-be823c44-fa65-4e2f-b083-7b3c9126bf10'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:35.661506 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-be823c44-fa65-4e2f-b083-7b3c9126bf10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a46b95f5568488689be997d34553184', 'request_id': 'req-be823c44-fa65-4e2f-b083-7b3c9126bf10'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:35.681255 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-be823c44-fa65-4e2f-b083-7b3c9126bf10 admin admin] The token cache is being invalidate because project 3a46b95f5568488689be997d34553184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:35.682616 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-be823c44-fa65-4e2f-b083-7b3c9126bf10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:35.686860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-be823c44-fa65-4e2f-b083-7b3c9126bf10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:35.689150 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1311/2619] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:35 2022] DELETE /identity/v3/projects/3a46b95f5568488689be997d34553184 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:35.696427 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1c96f-30f7-4be7-a5a9-95ceda8e1c7a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:35.697356 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1c96f-30f7-4be7-a5a9-95ceda8e1c7a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:35.698223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1c96f-30f7-4be7-a5a9-95ceda8e1c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:35.736634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-49c1c96f-30f7-4be7-a5a9-95ceda8e1c7a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:35.739051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-49c1c96f-30f7-4be7-a5a9-95ceda8e1c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:35.815710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1309/2620] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:35.824931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2d9b2-0f85-42d2-be3b-ef36a5f0c6c6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:35.825237 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2d9b2-0f85-42d2-be3b-ef36a5f0c6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:35.825496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2d9b2-0f85-42d2-be3b-ef36a5f0c6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:35.848751 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4cb2d9b2-0f85-42d2-be3b-ef36a5f0c6c6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:35.850035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4cb2d9b2-0f85-42d2-be3b-ef36a5f0c6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:35.859149 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1312/2621] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:35.865697 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1c96f-30f7-4be7-a5a9-95ceda8e1c7a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:35.877612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1c96f-30f7-4be7-a5a9-95ceda8e1c7a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:35.883652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617120a-9715-4f25-bf45-e62a263bd2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:35.884507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617120a-9715-4f25-bf45-e62a263bd2cc admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:35.885176 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617120a-9715-4f25-bf45-e62a263bd2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:35.885176 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617120a-9715-4f25-bf45-e62a263bd2cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:35.885813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617120a-9715-4f25-bf45-e62a263bd2cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:35.886127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617120a-9715-4f25-bf45-e62a263bd2cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:35.887844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617120a-9715-4f25-bf45-e62a263bd2cc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:35.903249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1310/2622] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:18:35.911121 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2d9b2-0f85-42d2-be3b-ef36a5f0c6c6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:35.930376 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2d9b2-0f85-42d2-be3b-ef36a5f0c6c6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:35.942206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a0570-0b03-4623-90aa-e7fe24c10ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:35.943407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a0570-0b03-4623-90aa-e7fe24c10ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:35.943970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a0570-0b03-4623-90aa-e7fe24c10ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:35.944466 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a0570-0b03-4623-90aa-e7fe24c10ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:35.945805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a0570-0b03-4623-90aa-e7fe24c10ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:35.949693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a0570-0b03-4623-90aa-e7fe24c10ce6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:35.971741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:36.014649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1313/2623] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:35 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:36.019912 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617120a-9715-4f25-bf45-e62a263bd2cc admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.035777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617120a-9715-4f25-bf45-e62a263bd2cc admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.041144 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b3cf4-4e82-459b-8646-f24690257b22 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.041627 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b3cf4-4e82-459b-8646-f24690257b22 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.042074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b3cf4-4e82-459b-8646-f24690257b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.075969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:36.077406 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b65b3cf4-4e82-459b-8646-f24690257b22 None None] MFA Rules not processed for user `d5f1382ca00c491092affa569b0b9a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:36.078708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b65b3cf4-4e82-459b-8646-f24690257b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:36.114099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c65ccc-5290-4255-b553-9897dff8c113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.115079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c65ccc-5290-4255-b553-9897dff8c113 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.115688 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c65ccc-5290-4255-b553-9897dff8c113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.116134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c65ccc-5290-4255-b553-9897dff8c113 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.117068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c65ccc-5290-4255-b553-9897dff8c113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.119042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c65ccc-5290-4255-b553-9897dff8c113 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:36.146440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1311/2624] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:36 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:36.152318 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c65ccc-5290-4255-b553-9897dff8c113 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.166559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c65ccc-5290-4255-b553-9897dff8c113 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.172552 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3821ae-6b75-4dda-975d-68d1d5864532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.173248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3821ae-6b75-4dda-975d-68d1d5864532 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.173540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3821ae-6b75-4dda-975d-68d1d5864532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.173876 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3821ae-6b75-4dda-975d-68d1d5864532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.175134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3821ae-6b75-4dda-975d-68d1d5864532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.177054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3821ae-6b75-4dda-975d-68d1d5864532 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:36.188560 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1312/2625] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:36.196446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3821ae-6b75-4dda-975d-68d1d5864532 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.210228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3821ae-6b75-4dda-975d-68d1d5864532 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.216891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cb9c3-0317-4bf8-8c96-9f37f247456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.217860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cb9c3-0317-4bf8-8c96-9f37f247456b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.218149 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cb9c3-0317-4bf8-8c96-9f37f247456b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.218437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cb9c3-0317-4bf8-8c96-9f37f247456b admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.219234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cb9c3-0317-4bf8-8c96-9f37f247456b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.221917 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cb9c3-0317-4bf8-8c96-9f37f247456b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:36.232789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1314/2626] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:36.238199 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1313/2627] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:36.243213 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b3cf4-4e82-459b-8646-f24690257b22 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.246726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cb9c3-0317-4bf8-8c96-9f37f247456b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.249839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e56cb9c3-0317-4bf8-8c96-9f37f247456b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:36.263073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b3cf4-4e82-459b-8646-f24690257b22 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.272491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69702c-df91-48b4-93d8-ee09db9d9a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.274247 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69702c-df91-48b4-93d8-ee09db9d9a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.274827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69702c-df91-48b4-93d8-ee09db9d9a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.275460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69702c-df91-48b4-93d8-ee09db9d9a61 admin admin] PATH_INFO: `/v3/projects/429e991e1a8d41d3bc0e6ba6cd507b86/users/4ecf580393e1429aa8dfe2a9d535abd0/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.303012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69702c-df91-48b4-93d8-ee09db9d9a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=429e991e1a8d41d3bc0e6ba6cd507b86, user_id=4ecf580393e1429aa8dfe2a9d535abd0, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.308375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69702c-df91-48b4-93d8-ee09db9d9a61 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:36.336887 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1315/2628] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:36 2022] PUT /identity/v3/projects/429e991e1a8d41d3bc0e6ba6cd507b86/users/4ecf580393e1429aa8dfe2a9d535abd0/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:36.351156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950d7e0-3d0e-4145-a67e-2ca418ed66f2 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.351790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950d7e0-3d0e-4145-a67e-2ca418ed66f2 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.352322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950d7e0-3d0e-4145-a67e-2ca418ed66f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.354666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cb9c3-0317-4bf8-8c96-9f37f247456b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.400127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e950d7e0-3d0e-4145-a67e-2ca418ed66f2 None None] MFA Rules not processed for user `4ecf580393e1429aa8dfe2a9d535abd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:36.401579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e950d7e0-3d0e-4145-a67e-2ca418ed66f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:36.425119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c42ee3-ad6a-49c4-9983-8bb549ff83d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.425795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c42ee3-ad6a-49c4-9983-8bb549ff83d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.426084 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c42ee3-ad6a-49c4-9983-8bb549ff83d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.426388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c42ee3-ad6a-49c4-9983-8bb549ff83d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.428627 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-81c42ee3-ad6a-49c4-9983-8bb549ff83d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:36.501635 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1316/2629] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:36.507851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950d7e0-3d0e-4145-a67e-2ca418ed66f2 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.520108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c42ee3-ad6a-49c4-9983-8bb549ff83d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.521586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c42ee3-ad6a-49c4-9983-8bb549ff83d1 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:36.532806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1314/2630] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:36.564334 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:36.589898 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950d7e0-3d0e-4145-a67e-2ca418ed66f2 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.595256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ece982-61fe-4e60-b63f-48676af9c3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.596039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ece982-61fe-4e60-b63f-48676af9c3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.596960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ece982-61fe-4e60-b63f-48676af9c3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.597293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ece982-61fe-4e60-b63f-48676af9c3e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.597979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ece982-61fe-4e60-b63f-48676af9c3e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.599985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ece982-61fe-4e60-b63f-48676af9c3e5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:36.633051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1317/2631] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:36 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:36.637816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c42ee3-ad6a-49c4-9983-8bb549ff83d1 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.655727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c42ee3-ad6a-49c4-9983-8bb549ff83d1 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.664674 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:36.753047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327da8c-1ee4-4775-b70c-09b833b89613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.753047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327da8c-1ee4-4775-b70c-09b833b89613 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.753047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327da8c-1ee4-4775-b70c-09b833b89613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.753451 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327da8c-1ee4-4775-b70c-09b833b89613 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.753938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327da8c-1ee4-4775-b70c-09b833b89613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.756357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327da8c-1ee4-4775-b70c-09b833b89613 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:36.792034 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1315/2632] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:36 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:36.799911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ece982-61fe-4e60-b63f-48676af9c3e5 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.809401 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ece982-61fe-4e60-b63f-48676af9c3e5 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.816187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd03152-61b2-46fc-8706-d254b82af656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.817006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd03152-61b2-46fc-8706-d254b82af656 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.817364 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd03152-61b2-46fc-8706-d254b82af656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.817724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd03152-61b2-46fc-8706-d254b82af656 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.818557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd03152-61b2-46fc-8706-d254b82af656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.820317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd03152-61b2-46fc-8706-d254b82af656 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:36.833533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1318/2633] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:36.837703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327da8c-1ee4-4775-b70c-09b833b89613 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.849655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327da8c-1ee4-4775-b70c-09b833b89613 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:36.854816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e59552-16ed-4532-8e5c-448f0d15718a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:36.855889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e59552-16ed-4532-8e5c-448f0d15718a admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:36.856281 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e59552-16ed-4532-8e5c-448f0d15718a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:36.856756 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e59552-16ed-4532-8e5c-448f0d15718a admin admin] PATH_INFO: `/v3/projects/0f19fada591d42daac7e5b0764fd6de7/users/6251051e4f6245f589d7ffd75e75acb9/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:36.881900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e59552-16ed-4532-8e5c-448f0d15718a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f19fada591d42daac7e5b0764fd6de7, user_id=6251051e4f6245f589d7ffd75e75acb9, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:36.887614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e59552-16ed-4532-8e5c-448f0d15718a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:36.910519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1316/2634] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:36 2022] PUT /identity/v3/projects/0f19fada591d42daac7e5b0764fd6de7/users/6251051e4f6245f589d7ffd75e75acb9/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:36.914243 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd03152-61b2-46fc-8706-d254b82af656 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:36.931886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd03152-61b2-46fc-8706-d254b82af656 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:37.033127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a7ed7-6fdf-4723-85f6-02dbabd1d8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:37.034696 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a7ed7-6fdf-4723-85f6-02dbabd1d8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:37.037843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a7ed7-6fdf-4723-85f6-02dbabd1d8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:37.037843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a7ed7-6fdf-4723-85f6-02dbabd1d8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:37.037843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a7ed7-6fdf-4723-85f6-02dbabd1d8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:37.038601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a7ed7-6fdf-4723-85f6-02dbabd1d8d9 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:37.055758 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1319/2635] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:37.060082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e59552-16ed-4532-8e5c-448f0d15718a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:37.071261 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e59552-16ed-4532-8e5c-448f0d15718a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:37.076360 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fe1f6-618d-416a-9897-a54278d07cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:37.077278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fe1f6-618d-416a-9897-a54278d07cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:37.077559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fe1f6-618d-416a-9897-a54278d07cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:37.077836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fe1f6-618d-416a-9897-a54278d07cc2 admin admin] PATH_INFO: `/v3/projects/0f19fada591d42daac7e5b0764fd6de7/users/6251051e4f6245f589d7ffd75e75acb9/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:37.079989 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fe1f6-618d-416a-9897-a54278d07cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f19fada591d42daac7e5b0764fd6de7, user_id=6251051e4f6245f589d7ffd75e75acb9, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:37.083045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fe1f6-618d-416a-9897-a54278d07cc2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:37.098941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1317/2636] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:37 2022] PUT /identity/v3/projects/0f19fada591d42daac7e5b0764fd6de7/users/6251051e4f6245f589d7ffd75e75acb9/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:37.114784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d87831-0ca8-43e8-8aff-5a25ef985f2a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:37.116214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d87831-0ca8-43e8-8aff-5a25ef985f2a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:37.116995 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d87831-0ca8-43e8-8aff-5a25ef985f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:37.156118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-82d87831-0ca8-43e8-8aff-5a25ef985f2a None None] MFA Rules not processed for user `6251051e4f6245f589d7ffd75e75acb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:37.157759 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-82d87831-0ca8-43e8-8aff-5a25ef985f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:37.280391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1320/2637] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:37.294816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fe1f6-618d-416a-9897-a54278d07cc2 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:37.387780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fe1f6-618d-416a-9897-a54278d07cc2 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:37.393786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040691b-9d80-43b1-8ad4-22bccf7111a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:37.394970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040691b-9d80-43b1-8ad4-22bccf7111a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:37.395520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040691b-9d80-43b1-8ad4-22bccf7111a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:37.396006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040691b-9d80-43b1-8ad4-22bccf7111a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:37.413765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040691b-9d80-43b1-8ad4-22bccf7111a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:37.416540 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040691b-9d80-43b1-8ad4-22bccf7111a1 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:37.435960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1318/2638] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:38.042278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d87831-0ca8-43e8-8aff-5a25ef985f2a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:38.044020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-82d87831-0ca8-43e8-8aff-5a25ef985f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:38.060053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d87831-0ca8-43e8-8aff-5a25ef985f2a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:38.064168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39e472-3795-4dbd-acc0-45751139b782 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:38.065808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39e472-3795-4dbd-acc0-45751139b782 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:38.066271 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39e472-3795-4dbd-acc0-45751139b782 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:38.066716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39e472-3795-4dbd-acc0-45751139b782 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:38.115723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040691b-9d80-43b1-8ad4-22bccf7111a1 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:38.116776 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:38.158774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39e472-3795-4dbd-acc0-45751139b782 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:38.160786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39e472-3795-4dbd-acc0-45751139b782 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:38.173973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1321/2639] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:38.211798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040691b-9d80-43b1-8ad4-22bccf7111a1 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:38.217235 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:38.218184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:38.218445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:38.218762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] PATH_INFO: `/v3/users/4ecf580393e1429aa8dfe2a9d535abd0` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:38.220048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ecf580393e1429aa8dfe2a9d535abd0)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:38.222357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:38.231311 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:38.313551 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ecf580393e1429aa8dfe2a9d535abd0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.314452 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ecf580393e1429aa8dfe2a9d535abd0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.324168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ecf580393e1429aa8dfe2a9d535abd0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.335368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ecf580393e1429aa8dfe2a9d535abd0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.336374 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ecf580393e1429aa8dfe2a9d535abd0'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.368453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1319/2640] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:38 2022] DELETE /identity/v3/users/4ecf580393e1429aa8dfe2a9d535abd0 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:38.373686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca431516-8c46-4ac4-beb0-5ccc7ec1759b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:38.374043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca431516-8c46-4ac4-beb0-5ccc7ec1759b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:38.374474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca431516-8c46-4ac4-beb0-5ccc7ec1759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:38.403089 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ca431516-8c46-4ac4-beb0-5ccc7ec1759b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:38.404782 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ca431516-8c46-4ac4-beb0-5ccc7ec1759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:38.502877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1322/2641] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:38.526780 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:38.529178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:38.618508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da1e24-483e-44f2-bd9c-2b7c613cd77b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:38.627604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f5cac-795f-42d1-8faf-a072fdd39598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:38.627604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f5cac-795f-42d1-8faf-a072fdd39598 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:38.627604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f5cac-795f-42d1-8faf-a072fdd39598 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:38.627604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f5cac-795f-42d1-8faf-a072fdd39598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:38.641429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f5cac-795f-42d1-8faf-a072fdd39598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:38.643721 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f5cac-795f-42d1-8faf-a072fdd39598 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:38.657805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1320/2642] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:38.840188 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca431516-8c46-4ac4-beb0-5ccc7ec1759b None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:38.841804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ca431516-8c46-4ac4-beb0-5ccc7ec1759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:38.856932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca431516-8c46-4ac4-beb0-5ccc7ec1759b None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:38.865970 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:38.867842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:38.868438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:38.869225 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] PATH_INFO: `/v3/users/2d32aa78535b4dbb8bbc15139576284a` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:38.875272 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d32aa78535b4dbb8bbc15139576284a)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:38.887955 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:38.964861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d32aa78535b4dbb8bbc15139576284a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.975875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d32aa78535b4dbb8bbc15139576284a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.976770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d32aa78535b4dbb8bbc15139576284a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.987539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d32aa78535b4dbb8bbc15139576284a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:38.988419 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d32aa78535b4dbb8bbc15139576284a'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.014615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1323/2643] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:38 2022] DELETE /identity/v3/users/2d32aa78535b4dbb8bbc15139576284a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:39.020723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3eee7-b9b7-4a3e-b712-ed02c193e579 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:39.021122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3eee7-b9b7-4a3e-b712-ed02c193e579 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:39.021417 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3eee7-b9b7-4a3e-b712-ed02c193e579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:39.041839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-e1c3eee7-b9b7-4a3e-b712-ed02c193e579 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:39.043102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e1c3eee7-b9b7-4a3e-b712-ed02c193e579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:39.051597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:39.053256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:39.120360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:39.145336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1321/2644] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:39.156949 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f63f1-336b-4ac0-a291-521f03e97a33 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:39.159547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3eee7-b9b7-4a3e-b712-ed02c193e579 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:39.161919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e1c3eee7-b9b7-4a3e-b712-ed02c193e579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:39.164571 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:39.165839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:39.166135 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:39.166448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] PATH_INFO: `/v3/projects/429e991e1a8d41d3bc0e6ba6cd507b86` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:39.167767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=429e991e1a8d41d3bc0e6ba6cd507b86)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:39.169721 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:39.209802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:39.233235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '429e991e1a8d41d3bc0e6ba6cd507b86', 'request_id': 'req-d431326e-0554-4da5-bf74-8798beb91606'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.242115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3eee7-b9b7-4a3e-b712-ed02c193e579 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:39.245134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '429e991e1a8d41d3bc0e6ba6cd507b86', 'request_id': 'req-d431326e-0554-4da5-bf74-8798beb91606'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.253281 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '429e991e1a8d41d3bc0e6ba6cd507b86', 'request_id': 'req-d431326e-0554-4da5-bf74-8798beb91606'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.273340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] The token cache is being invalidate because project 429e991e1a8d41d3bc0e6ba6cd507b86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:39.273964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.275474 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.277497 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1324/2645] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:39 2022] DELETE /identity/v3/projects/429e991e1a8d41d3bc0e6ba6cd507b86 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:39.282115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:39.284102 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:39.308468 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761a328-c6fb-4b3f-8146-d3cbc95d74d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:39.309707 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761a328-c6fb-4b3f-8146-d3cbc95d74d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:39.310186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761a328-c6fb-4b3f-8146-d3cbc95d74d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:39.310547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761a328-c6fb-4b3f-8146-d3cbc95d74d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:39.313227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3761a328-c6fb-4b3f-8146-d3cbc95d74d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:39.361148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431326e-0554-4da5-bf74-8798beb91606 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:39.366292 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:39.367198 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:39.367522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:39.367808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] PATH_INFO: `/v3/users/6251051e4f6245f589d7ffd75e75acb9` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:39.369122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6251051e4f6245f589d7ffd75e75acb9)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:39.371257 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:39.427028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6251051e4f6245f589d7ffd75e75acb9'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.436381 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761a328-c6fb-4b3f-8146-d3cbc95d74d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:39.436711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6251051e4f6245f589d7ffd75e75acb9'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.437466 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6251051e4f6245f589d7ffd75e75acb9'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.439397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761a328-c6fb-4b3f-8146-d3cbc95d74d6 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:39.446982 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6251051e4f6245f589d7ffd75e75acb9'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.447827 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6251051e4f6245f589d7ffd75e75acb9'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:39.460341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1322/2646] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:39.469391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1325/2647] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:39 2022] DELETE /identity/v3/users/6251051e4f6245f589d7ffd75e75acb9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:39.470022 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:39.897146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761a328-c6fb-4b3f-8146-d3cbc95d74d6 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:39.898613 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3761a328-c6fb-4b3f-8146-d3cbc95d74d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:39.913428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761a328-c6fb-4b3f-8146-d3cbc95d74d6 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:39.956186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:39.957991 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:40.041810 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:40.043622 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.044439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.045021 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] PATH_INFO: `/v3/projects/12257ef0c75c447c8dc88488fe119db3` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.048479 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] RBAC: Authorizing `identity:delete_project(project_id=12257ef0c75c447c8dc88488fe119db3)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:40.053502 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:40.082061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facae468-5656-425c-acb8-2c1a405605f0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:40.087803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877dea74-4dab-4a2f-b845-45b1d661613d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:40.089165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877dea74-4dab-4a2f-b845-45b1d661613d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.089701 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877dea74-4dab-4a2f-b845-45b1d661613d admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.090091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877dea74-4dab-4a2f-b845-45b1d661613d admin admin] PATH_INFO: `/v3/projects/0f19fada591d42daac7e5b0764fd6de7` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.093255 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877dea74-4dab-4a2f-b845-45b1d661613d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f19fada591d42daac7e5b0764fd6de7)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:40.097108 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877dea74-4dab-4a2f-b845-45b1d661613d admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:40.151102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:40.175547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12257ef0c75c447c8dc88488fe119db3', 'request_id': 'req-2c0289ea-718b-47fc-9b30-1c8683f1337b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.187620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12257ef0c75c447c8dc88488fe119db3', 'request_id': 'req-2c0289ea-718b-47fc-9b30-1c8683f1337b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.202079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12257ef0c75c447c8dc88488fe119db3', 'request_id': 'req-2c0289ea-718b-47fc-9b30-1c8683f1337b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.223495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] The token cache is being invalidate because project 12257ef0c75c447c8dc88488fe119db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:40.223495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.225632 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.227819 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1323/2648] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:39 2022] DELETE /identity/v3/projects/12257ef0c75c447c8dc88488fe119db3 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:40.231053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-877dea74-4dab-4a2f-b845-45b1d661613d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f19fada591d42daac7e5b0764fd6de7', 'request_id': 'req-877dea74-4dab-4a2f-b845-45b1d661613d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.232438 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:40.234359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:40.242293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-877dea74-4dab-4a2f-b845-45b1d661613d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f19fada591d42daac7e5b0764fd6de7', 'request_id': 'req-877dea74-4dab-4a2f-b845-45b1d661613d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.253717 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-877dea74-4dab-4a2f-b845-45b1d661613d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f19fada591d42daac7e5b0764fd6de7', 'request_id': 'req-877dea74-4dab-4a2f-b845-45b1d661613d'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.271922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-877dea74-4dab-4a2f-b845-45b1d661613d admin admin] The token cache is being invalidate because project 0f19fada591d42daac7e5b0764fd6de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:40.272344 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-877dea74-4dab-4a2f-b845-45b1d661613d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.273311 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-877dea74-4dab-4a2f-b845-45b1d661613d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.275103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1326/2649] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:39 2022] DELETE /identity/v3/projects/0f19fada591d42daac7e5b0764fd6de7 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:40.280961 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5337e41-e812-4787-9012-9739a2185c5e None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.281457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5337e41-e812-4787-9012-9739a2185c5e None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.281903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5337e41-e812-4787-9012-9739a2185c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.308481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b5337e41-e812-4787-9012-9739a2185c5e None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:40.310088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b5337e41-e812-4787-9012-9739a2185c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:40.321761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0289ea-718b-47fc-9b30-1c8683f1337b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:40.378847 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1327/2650] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:40.388345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45fc9e-1835-475d-822f-8e025e1b6223 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.388638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45fc9e-1835-475d-822f-8e025e1b6223 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.388928 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45fc9e-1835-475d-822f-8e025e1b6223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.408770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bd45fc9e-1835-475d-822f-8e025e1b6223 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:40.410369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bd45fc9e-1835-475d-822f-8e025e1b6223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:40.414855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:40.415802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.416088 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.416372 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] PATH_INFO: `/v3/users/b381872678754ca3ae6ff11a06ad9e3c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.417687 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b381872678754ca3ae6ff11a06ad9e3c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:40.419732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:40.420417 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1328/2651] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:40.426715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45fc9e-1835-475d-822f-8e025e1b6223 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:40.444675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45fc9e-1835-475d-822f-8e025e1b6223 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:40.452879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558c34e-ef07-4150-873c-164031779b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:40.453575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558c34e-ef07-4150-873c-164031779b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.453913 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558c34e-ef07-4150-873c-164031779b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.454146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558c34e-ef07-4150-873c-164031779b1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.454896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558c34e-ef07-4150-873c-164031779b1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:40.455232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558c34e-ef07-4150-873c-164031779b1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:40.456988 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558c34e-ef07-4150-873c-164031779b1c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:40.472345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1329/2652] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:18:40.476913 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558c34e-ef07-4150-873c-164031779b1c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:40.481250 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b381872678754ca3ae6ff11a06ad9e3c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.481965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b381872678754ca3ae6ff11a06ad9e3c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.484120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:40.490219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558c34e-ef07-4150-873c-164031779b1c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:40.491462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b381872678754ca3ae6ff11a06ad9e3c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.499406 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1369f3d-6ac9-4db0-8731-ac2aa0579bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:40.500600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1369f3d-6ac9-4db0-8731-ac2aa0579bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.501626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b381872678754ca3ae6ff11a06ad9e3c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.501837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1369f3d-6ac9-4db0-8731-ac2aa0579bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.503053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1369f3d-6ac9-4db0-8731-ac2aa0579bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.503177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b381872678754ca3ae6ff11a06ad9e3c'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:40.505082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1369f3d-6ac9-4db0-8731-ac2aa0579bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:40.509774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1369f3d-6ac9-4db0-8731-ac2aa0579bc0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:40.515608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:40.531097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1324/2653] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:40 2022] DELETE /identity/v3/users/b381872678754ca3ae6ff11a06ad9e3c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:40.561691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1330/2654] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:40 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:40.568189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:40.570624 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:40.675910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f7164-3a02-499c-867c-a46437dd19a1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:40.784578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eb76c-091a-417c-bd95-671e2393b285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:40.785887 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eb76c-091a-417c-bd95-671e2393b285 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.786388 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eb76c-091a-417c-bd95-671e2393b285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.786872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eb76c-091a-417c-bd95-671e2393b285 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.788234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8eb76c-091a-417c-bd95-671e2393b285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:40.792523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8eb76c-091a-417c-bd95-671e2393b285 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:40.826313 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1325/2655] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:40 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:40.833093 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1369f3d-6ac9-4db0-8731-ac2aa0579bc0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:40.852540 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1369f3d-6ac9-4db0-8731-ac2aa0579bc0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:40.860733 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb2f98-a582-4004-8bfb-9dd315c586a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:40.862715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb2f98-a582-4004-8bfb-9dd315c586a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.863105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb2f98-a582-4004-8bfb-9dd315c586a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.863669 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb2f98-a582-4004-8bfb-9dd315c586a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.865128 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb2f98-a582-4004-8bfb-9dd315c586a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:40.868194 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb2f98-a582-4004-8bfb-9dd315c586a2 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:40.887427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1331/2656] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:40.893964 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eb76c-091a-417c-bd95-671e2393b285 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:40.907500 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eb76c-091a-417c-bd95-671e2393b285 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:40.914741 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e059bd-3f86-44a2-b298-69360a345a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:40.915648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e059bd-3f86-44a2-b298-69360a345a7e admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.915960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e059bd-3f86-44a2-b298-69360a345a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.916282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e059bd-3f86-44a2-b298-69360a345a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.917851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e059bd-3f86-44a2-b298-69360a345a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:40.919703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e059bd-3f86-44a2-b298-69360a345a7e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:40.939257 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1326/2657] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:40.944099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb2f98-a582-4004-8bfb-9dd315c586a2 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:40.944275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e059bd-3f86-44a2-b298-69360a345a7e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:40.944456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-38fb2f98-a582-4004-8bfb-9dd315c586a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:40.963842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e059bd-3f86-44a2-b298-69360a345a7e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:40.973969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c78b9-fc24-44f9-8bb1-322418d7d0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:40.975650 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c78b9-fc24-44f9-8bb1-322418d7d0ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.976193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c78b9-fc24-44f9-8bb1-322418d7d0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.976705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c78b9-fc24-44f9-8bb1-322418d7d0ef admin admin] PATH_INFO: `/v3/projects/5b4c3918e2cb4426a85a2e0f18e508f8/users/c11b900fd6e54ff7b4fb0b23c6482afc/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.979775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb2f98-a582-4004-8bfb-9dd315c586a2 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:40.992584 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32df04-464e-4439-8473-e5bfa8c4724f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:40.994709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32df04-464e-4439-8473-e5bfa8c4724f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:40.995111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32df04-464e-4439-8473-e5bfa8c4724f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:40.995601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32df04-464e-4439-8473-e5bfa8c4724f admin admin] PATH_INFO: `/v3/projects/50e5302ec83844f080a0ac1ed592a617` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:40.997487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32df04-464e-4439-8473-e5bfa8c4724f admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e5302ec83844f080a0ac1ed592a617)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.000313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32df04-464e-4439-8473-e5bfa8c4724f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.008147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c78b9-fc24-44f9-8bb1-322418d7d0ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b4c3918e2cb4426a85a2e0f18e508f8, user_id=c11b900fd6e54ff7b4fb0b23c6482afc, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.013796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c78b9-fc24-44f9-8bb1-322418d7d0ef admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.038001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1327/2658] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:40 2022] PUT /identity/v3/projects/5b4c3918e2cb4426a85a2e0f18e508f8/users/c11b900fd6e54ff7b4fb0b23c6482afc/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:41.057359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220a754-5fd2-4a4a-9570-fce74136f7bd None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.057863 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220a754-5fd2-4a4a-9570-fce74136f7bd None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.058375 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220a754-5fd2-4a4a-9570-fce74136f7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.069465 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f32df04-464e-4439-8473-e5bfa8c4724f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e5302ec83844f080a0ac1ed592a617', 'request_id': 'req-2f32df04-464e-4439-8473-e5bfa8c4724f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:41.077968 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f32df04-464e-4439-8473-e5bfa8c4724f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e5302ec83844f080a0ac1ed592a617', 'request_id': 'req-2f32df04-464e-4439-8473-e5bfa8c4724f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:41.085448 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f32df04-464e-4439-8473-e5bfa8c4724f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e5302ec83844f080a0ac1ed592a617', 'request_id': 'req-2f32df04-464e-4439-8473-e5bfa8c4724f'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:41.103144 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f32df04-464e-4439-8473-e5bfa8c4724f admin admin] The token cache is being invalidate because project 50e5302ec83844f080a0ac1ed592a617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:41.103610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f32df04-464e-4439-8473-e5bfa8c4724f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:41.104401 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2f32df04-464e-4439-8473-e5bfa8c4724f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:41.106318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1332/2659] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:40 2022] DELETE /identity/v3/projects/50e5302ec83844f080a0ac1ed592a617 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:41.106459 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2220a754-5fd2-4a4a-9570-fce74136f7bd None None] MFA Rules not processed for user `c11b900fd6e54ff7b4fb0b23c6482afc`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:41.108554 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2220a754-5fd2-4a4a-9570-fce74136f7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:41.112303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca165b0-38ad-4bc4-939f-44317f8eaa79 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.112638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca165b0-38ad-4bc4-939f-44317f8eaa79 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.112956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca165b0-38ad-4bc4-939f-44317f8eaa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.134787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4ca165b0-38ad-4bc4-939f-44317f8eaa79 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:41.137129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4ca165b0-38ad-4bc4-939f-44317f8eaa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:41.178878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:41.220228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1328/2660] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:41.245432 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220a754-5fd2-4a4a-9570-fce74136f7bd None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.247467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2220a754-5fd2-4a4a-9570-fce74136f7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:41.253451 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1333/2661] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:41.263573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b982-aa7d-4e24-b867-334f849c389a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.264129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b982-aa7d-4e24-b867-334f849c389a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.264672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b982-aa7d-4e24-b867-334f849c389a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.300459 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8b44b982-aa7d-4e24-b867-334f849c389a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:41.301507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8b44b982-aa7d-4e24-b867-334f849c389a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:41.318808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1334/2662] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:41.320582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b982-aa7d-4e24-b867-334f849c389a None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.332536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220a754-5fd2-4a4a-9570-fce74136f7bd None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.335461 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b982-aa7d-4e24-b867-334f849c389a None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.336818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64197f15-0a3f-4fd4-8333-e420391e8ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.337713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64197f15-0a3f-4fd4-8333-e420391e8ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.338301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64197f15-0a3f-4fd4-8333-e420391e8ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.338645 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64197f15-0a3f-4fd4-8333-e420391e8ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.343024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986bb33-e7fd-4552-9bfc-315bb8ef9478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.344020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986bb33-e7fd-4552-9bfc-315bb8ef9478 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.344477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986bb33-e7fd-4552-9bfc-315bb8ef9478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.344864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986bb33-e7fd-4552-9bfc-315bb8ef9478 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.346186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986bb33-e7fd-4552-9bfc-315bb8ef9478 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:41.346706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986bb33-e7fd-4552-9bfc-315bb8ef9478 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.349352 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64197f15-0a3f-4fd4-8333-e420391e8ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.349611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986bb33-e7fd-4552-9bfc-315bb8ef9478 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.351256 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64197f15-0a3f-4fd4-8333-e420391e8ff6 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.363222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1329/2663] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:18:41.373741 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1335/2664] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:18:41.379467 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64197f15-0a3f-4fd4-8333-e420391e8ff6 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.398680 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64197f15-0a3f-4fd4-8333-e420391e8ff6 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.406639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e2b80-4f8f-4f0f-8fe4-37d2c944c81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.407766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e2b80-4f8f-4f0f-8fe4-37d2c944c81b admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.408252 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e2b80-4f8f-4f0f-8fe4-37d2c944c81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.409311 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e2b80-4f8f-4f0f-8fe4-37d2c944c81b admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.410546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e2b80-4f8f-4f0f-8fe4-37d2c944c81b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.414247 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e2b80-4f8f-4f0f-8fe4-37d2c944c81b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.453229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1330/2665] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:41 2022] POST /identity/v3/projects => generated 393 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:41.458424 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986bb33-e7fd-4552-9bfc-315bb8ef9478 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.476990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986bb33-e7fd-4552-9bfc-315bb8ef9478 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.539680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:41.590982 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d69381-0bad-44ba-897c-9d2841d1b813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.592372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d69381-0bad-44ba-897c-9d2841d1b813 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.592749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d69381-0bad-44ba-897c-9d2841d1b813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.593091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d69381-0bad-44ba-897c-9d2841d1b813 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.594056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d69381-0bad-44ba-897c-9d2841d1b813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.596646 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d69381-0bad-44ba-897c-9d2841d1b813 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.619744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1336/2666] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:41 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:41.624022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e2b80-4f8f-4f0f-8fe4-37d2c944c81b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.635363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e2b80-4f8f-4f0f-8fe4-37d2c944c81b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.642302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec1eba-18b3-40a9-8dbe-7adf8ebabf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.643648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec1eba-18b3-40a9-8dbe-7adf8ebabf58 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.644082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec1eba-18b3-40a9-8dbe-7adf8ebabf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.644347 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec1eba-18b3-40a9-8dbe-7adf8ebabf58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.645297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec1eba-18b3-40a9-8dbe-7adf8ebabf58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.646954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec1eba-18b3-40a9-8dbe-7adf8ebabf58 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.667954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1331/2667] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:41.673200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d69381-0bad-44ba-897c-9d2841d1b813 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.688439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d69381-0bad-44ba-897c-9d2841d1b813 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.700642 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d3799-2342-4991-af41-f6375e29e527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.702076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d3799-2342-4991-af41-f6375e29e527 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.702746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d3799-2342-4991-af41-f6375e29e527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.703476 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d3799-2342-4991-af41-f6375e29e527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.705207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d3799-2342-4991-af41-f6375e29e527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.709376 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d3799-2342-4991-af41-f6375e29e527 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.729626 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1337/2668] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:41.734805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec1eba-18b3-40a9-8dbe-7adf8ebabf58 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:41.765885 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec1eba-18b3-40a9-8dbe-7adf8ebabf58 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:41.775512 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8441681-9839-4fb5-a769-82ccc62568d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:41.777583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8441681-9839-4fb5-a769-82ccc62568d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.778233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8441681-9839-4fb5-a769-82ccc62568d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.778963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8441681-9839-4fb5-a769-82ccc62568d8 admin admin] PATH_INFO: `/v3/projects/f42c151f821547968dd6ef65b8e9c3d2/users/a96aa50932fe49438699b05ff3a45059/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.812314 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8441681-9839-4fb5-a769-82ccc62568d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42c151f821547968dd6ef65b8e9c3d2, user_id=a96aa50932fe49438699b05ff3a45059, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:41.819458 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8441681-9839-4fb5-a769-82ccc62568d8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:41.837531 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1332/2669] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:41 2022] PUT /identity/v3/projects/f42c151f821547968dd6ef65b8e9c3d2/users/a96aa50932fe49438699b05ff3a45059/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:41.849003 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41a989-976b-4732-bb82-96acf846eb30 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:41.849326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41a989-976b-4732-bb82-96acf846eb30 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:41.849559 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41a989-976b-4732-bb82-96acf846eb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:41.881681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-6f41a989-976b-4732-bb82-96acf846eb30 None None] MFA Rules not processed for user `a96aa50932fe49438699b05ff3a45059`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:41.883130 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-6f41a989-976b-4732-bb82-96acf846eb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:41.957904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1338/2670] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:41.963104 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8441681-9839-4fb5-a769-82ccc62568d8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.039216 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8441681-9839-4fb5-a769-82ccc62568d8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.044267 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37f097-07d7-4e87-b6ae-6ebb1804d7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.044943 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37f097-07d7-4e87-b6ae-6ebb1804d7ba admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.045228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37f097-07d7-4e87-b6ae-6ebb1804d7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.045504 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37f097-07d7-4e87-b6ae-6ebb1804d7ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.046201 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37f097-07d7-4e87-b6ae-6ebb1804d7ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.048168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37f097-07d7-4e87-b6ae-6ebb1804d7ba admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.088851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1333/2671] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:41 2022] POST /identity/v3/projects => generated 393 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:42.094978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41a989-976b-4732-bb82-96acf846eb30 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.107703 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41a989-976b-4732-bb82-96acf846eb30 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.194519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3260cd-53ac-489e-9670-5d08c977ebe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.195952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3260cd-53ac-489e-9670-5d08c977ebe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.196498 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3260cd-53ac-489e-9670-5d08c977ebe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.196976 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3260cd-53ac-489e-9670-5d08c977ebe6 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.198276 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3260cd-53ac-489e-9670-5d08c977ebe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.201999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3260cd-53ac-489e-9670-5d08c977ebe6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.225207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1339/2672] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:42 2022] POST /identity/v3/users => generated 303 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:42.231005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37f097-07d7-4e87-b6ae-6ebb1804d7ba admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.243696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37f097-07d7-4e87-b6ae-6ebb1804d7ba admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.249094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a35546-adc7-4ef1-a976-b3141175d8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.249848 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a35546-adc7-4ef1-a976-b3141175d8ed admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.250258 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a35546-adc7-4ef1-a976-b3141175d8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.250620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a35546-adc7-4ef1-a976-b3141175d8ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.251536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a35546-adc7-4ef1-a976-b3141175d8ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.253440 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a35546-adc7-4ef1-a976-b3141175d8ed admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.259700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:42.268785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1334/2673] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:42.272436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3260cd-53ac-489e-9670-5d08c977ebe6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.284775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3260cd-53ac-489e-9670-5d08c977ebe6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.290230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289d7ad-8edb-4b27-8102-b3814497dd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.291222 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289d7ad-8edb-4b27-8102-b3814497dd5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.291578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289d7ad-8edb-4b27-8102-b3814497dd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.291899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289d7ad-8edb-4b27-8102-b3814497dd5a admin admin] PATH_INFO: `/v3/projects/4018179b5a6f467ba624542abbf47716/users/e00c1231d9df42bd8baff4e9fdbb52ed/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.313499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289d7ad-8edb-4b27-8102-b3814497dd5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4018179b5a6f467ba624542abbf47716, user_id=e00c1231d9df42bd8baff4e9fdbb52ed, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.316604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289d7ad-8edb-4b27-8102-b3814497dd5a admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.333153 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1340/2674] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:42 2022] PUT /identity/v3/projects/4018179b5a6f467ba624542abbf47716/users/e00c1231d9df42bd8baff4e9fdbb52ed/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:42.342226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a35546-adc7-4ef1-a976-b3141175d8ed admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.358104 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a35546-adc7-4ef1-a976-b3141175d8ed admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.431609 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcbe49-2150-40d4-b763-8ab9a089912b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.432310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcbe49-2150-40d4-b763-8ab9a089912b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.432595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcbe49-2150-40d4-b763-8ab9a089912b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.432871 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcbe49-2150-40d4-b763-8ab9a089912b admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.433619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcbe49-2150-40d4-b763-8ab9a089912b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.435282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcbe49-2150-40d4-b763-8ab9a089912b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.447443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1335/2675] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:42.453053 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289d7ad-8edb-4b27-8102-b3814497dd5a admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.466299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289d7ad-8edb-4b27-8102-b3814497dd5a admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.474642 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3e91d-cbe9-44a1-af64-a3fc912488b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.476240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3e91d-cbe9-44a1-af64-a3fc912488b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.476613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3e91d-cbe9-44a1-af64-a3fc912488b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.477041 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3e91d-cbe9-44a1-af64-a3fc912488b1 admin admin] PATH_INFO: `/v3/projects/4018179b5a6f467ba624542abbf47716/users/e00c1231d9df42bd8baff4e9fdbb52ed/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.480458 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3e91d-cbe9-44a1-af64-a3fc912488b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4018179b5a6f467ba624542abbf47716, user_id=e00c1231d9df42bd8baff4e9fdbb52ed, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.485525 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3e91d-cbe9-44a1-af64-a3fc912488b1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.492168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:42.506604 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1341/2676] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:42 2022] PUT /identity/v3/projects/4018179b5a6f467ba624542abbf47716/users/e00c1231d9df42bd8baff4e9fdbb52ed/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:42.518595 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29b5b0-3938-4b4c-bf40-b506e27cb3b3 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.518955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29b5b0-3938-4b4c-bf40-b506e27cb3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.519249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29b5b0-3938-4b4c-bf40-b506e27cb3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.559705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5d29b5b0-3938-4b4c-bf40-b506e27cb3b3 None None] MFA Rules not processed for user `e00c1231d9df42bd8baff4e9fdbb52ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:42.561898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5d29b5b0-3938-4b4c-bf40-b506e27cb3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:42.677370 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1336/2677] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:42.689377 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3e91d-cbe9-44a1-af64-a3fc912488b1 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.780337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3e91d-cbe9-44a1-af64-a3fc912488b1 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.787055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa326bcf-195b-480b-bd01-f285dabc1880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.788140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa326bcf-195b-480b-bd01-f285dabc1880 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.788606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa326bcf-195b-480b-bd01-f285dabc1880 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.789069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa326bcf-195b-480b-bd01-f285dabc1880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:42.804922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa326bcf-195b-480b-bd01-f285dabc1880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:42.807365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa326bcf-195b-480b-bd01-f285dabc1880 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:42.823678 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1342/2678] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:42.960744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29b5b0-3938-4b4c-bf40-b506e27cb3b3 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:42.962446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5d29b5b0-3938-4b4c-bf40-b506e27cb3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:42.982323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29b5b0-3938-4b4c-bf40-b506e27cb3b3 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:42.989896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b0823-e8b6-4aec-a113-d0a3131cebe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:42.990890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b0823-e8b6-4aec-a113-d0a3131cebe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:42.991583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b0823-e8b6-4aec-a113-d0a3131cebe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:42.991949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b0823-e8b6-4aec-a113-d0a3131cebe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:43.105919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b0823-e8b6-4aec-a113-d0a3131cebe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:43.107498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b0823-e8b6-4aec-a113-d0a3131cebe3 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:43.118852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1337/2679] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:44.989158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa326bcf-195b-480b-bd01-f285dabc1880 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:44.990694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-aa326bcf-195b-480b-bd01-f285dabc1880 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:45.067650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa326bcf-195b-480b-bd01-f285dabc1880 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:45.074613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:45.076090 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:45.076548 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:45.077000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] PATH_INFO: `/v3/users/c11b900fd6e54ff7b4fb0b23c6482afc` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:45.079039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c11b900fd6e54ff7b4fb0b23c6482afc)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:45.083038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:45.152139 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c11b900fd6e54ff7b4fb0b23c6482afc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.161978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c11b900fd6e54ff7b4fb0b23c6482afc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.162852 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c11b900fd6e54ff7b4fb0b23c6482afc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.177494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c11b900fd6e54ff7b4fb0b23c6482afc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.178478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c11b900fd6e54ff7b4fb0b23c6482afc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.218315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1343/2680] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:44 2022] DELETE /identity/v3/users/c11b900fd6e54ff7b4fb0b23c6482afc => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:45.224445 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50a1bb-a77e-4ff7-a1e9-da9867234f89 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:45.225325 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50a1bb-a77e-4ff7-a1e9-da9867234f89 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:45.225325 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50a1bb-a77e-4ff7-a1e9-da9867234f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:45.245385 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3e50a1bb-a77e-4ff7-a1e9-da9867234f89 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:45.246835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3e50a1bb-a77e-4ff7-a1e9-da9867234f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:45.263648 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:45.333133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1338/2681] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:45.343709 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:45.345227 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:45.369073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:45.416563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541140f-22d3-4490-8be0-70d5c0b61e00 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:45.420443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5facfa16-a83c-4cc4-81ea-a416183c9e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:45.421119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facfa16-a83c-4cc4-81ea-a416183c9e70 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:45.421405 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facfa16-a83c-4cc4-81ea-a416183c9e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:45.421690 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facfa16-a83c-4cc4-81ea-a416183c9e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:45.432628 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5facfa16-a83c-4cc4-81ea-a416183c9e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:45.434126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5facfa16-a83c-4cc4-81ea-a416183c9e70 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:45.447341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1344/2682] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:45.858516 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50a1bb-a77e-4ff7-a1e9-da9867234f89 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:45.860184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3e50a1bb-a77e-4ff7-a1e9-da9867234f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:45.884115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50a1bb-a77e-4ff7-a1e9-da9867234f89 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:45.889614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397904d-d4d3-45fc-ae22-594295e6b7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:45.890524 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397904d-d4d3-45fc-ae22-594295e6b7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:45.890812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397904d-d4d3-45fc-ae22-594295e6b7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:45.891099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397904d-d4d3-45fc-ae22-594295e6b7d3 admin admin] PATH_INFO: `/v3/projects/5b4c3918e2cb4426a85a2e0f18e508f8` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:45.893224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397904d-d4d3-45fc-ae22-594295e6b7d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b4c3918e2cb4426a85a2e0f18e508f8)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:45.895545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397904d-d4d3-45fc-ae22-594295e6b7d3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:45.968897 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a397904d-d4d3-45fc-ae22-594295e6b7d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b4c3918e2cb4426a85a2e0f18e508f8', 'request_id': 'req-a397904d-d4d3-45fc-ae22-594295e6b7d3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.975932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a397904d-d4d3-45fc-ae22-594295e6b7d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b4c3918e2cb4426a85a2e0f18e508f8', 'request_id': 'req-a397904d-d4d3-45fc-ae22-594295e6b7d3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.982500 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a397904d-d4d3-45fc-ae22-594295e6b7d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b4c3918e2cb4426a85a2e0f18e508f8', 'request_id': 'req-a397904d-d4d3-45fc-ae22-594295e6b7d3'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.996342 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a397904d-d4d3-45fc-ae22-594295e6b7d3 admin admin] The token cache is being invalidate because project 5b4c3918e2cb4426a85a2e0f18e508f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:45.996641 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a397904d-d4d3-45fc-ae22-594295e6b7d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.997330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-a397904d-d4d3-45fc-ae22-594295e6b7d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:45.999038 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1339/2683] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:45 2022] DELETE /identity/v3/projects/5b4c3918e2cb4426a85a2e0f18e508f8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:46.004959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d2610-e858-46e3-aa7d-9aa0528ada56 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.005554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d2610-e858-46e3-aa7d-9aa0528ada56 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.005554 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d2610-e858-46e3-aa7d-9aa0528ada56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.025776 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-b41d2610-e858-46e3-aa7d-9aa0528ada56 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:46.027030 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-b41d2610-e858-46e3-aa7d-9aa0528ada56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:46.092742 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1345/2684] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:18:46.101787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7833a-630b-4c7e-9504-5eac7fe92c1a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.102075 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7833a-630b-4c7e-9504-5eac7fe92c1a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.102359 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7833a-630b-4c7e-9504-5eac7fe92c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.123244 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-24e7833a-630b-4c7e-9504-5eac7fe92c1a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:46.124606 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-24e7833a-630b-4c7e-9504-5eac7fe92c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:46.133221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1340/2685] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:46.138048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d2610-e858-46e3-aa7d-9aa0528ada56 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.148436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d2610-e858-46e3-aa7d-9aa0528ada56 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.153306 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747a3a6-7570-47b6-878f-8bbacb198490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.153916 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747a3a6-7570-47b6-878f-8bbacb198490 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.154200 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747a3a6-7570-47b6-878f-8bbacb198490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.154478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747a3a6-7570-47b6-878f-8bbacb198490 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.155233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747a3a6-7570-47b6-878f-8bbacb198490 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:46.155585 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747a3a6-7570-47b6-878f-8bbacb198490 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.157249 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747a3a6-7570-47b6-878f-8bbacb198490 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.171184 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1346/2686] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:18:46.176545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7833a-630b-4c7e-9504-5eac7fe92c1a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.186583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7833a-630b-4c7e-9504-5eac7fe92c1a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.192237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588af259-43c9-4e94-b480-797e60532a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.192952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588af259-43c9-4e94-b480-797e60532a9e admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.193237 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588af259-43c9-4e94-b480-797e60532a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.193515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588af259-43c9-4e94-b480-797e60532a9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.194223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588af259-43c9-4e94-b480-797e60532a9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.196207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588af259-43c9-4e94-b480-797e60532a9e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.229333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1341/2687] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:46 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:46.233711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747a3a6-7570-47b6-878f-8bbacb198490 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.245403 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747a3a6-7570-47b6-878f-8bbacb198490 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.313891 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3b873-d739-4520-9aaa-18f9a1433624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.314606 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3b873-d739-4520-9aaa-18f9a1433624 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.315504 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3b873-d739-4520-9aaa-18f9a1433624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.315786 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3b873-d739-4520-9aaa-18f9a1433624 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.316496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3b873-d739-4520-9aaa-18f9a1433624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.318456 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3b873-d739-4520-9aaa-18f9a1433624 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.341356 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1347/2688] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:46 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:46.348699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588af259-43c9-4e94-b480-797e60532a9e admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.368510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588af259-43c9-4e94-b480-797e60532a9e admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.376794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbffd9c-9d9d-4423-9bfa-e311253badbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.378518 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbffd9c-9d9d-4423-9bfa-e311253badbb admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.379035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbffd9c-9d9d-4423-9bfa-e311253badbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.379678 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbffd9c-9d9d-4423-9bfa-e311253badbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.381145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbffd9c-9d9d-4423-9bfa-e311253badbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.384505 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbffd9c-9d9d-4423-9bfa-e311253badbb admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.406171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1342/2689] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:46.410487 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3b873-d739-4520-9aaa-18f9a1433624 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.421740 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3b873-d739-4520-9aaa-18f9a1433624 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.427271 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b98db5-0440-423a-9cc8-d9dee0206a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.427997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b98db5-0440-423a-9cc8-d9dee0206a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.428285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b98db5-0440-423a-9cc8-d9dee0206a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.428564 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b98db5-0440-423a-9cc8-d9dee0206a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.429556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b98db5-0440-423a-9cc8-d9dee0206a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.431208 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b98db5-0440-423a-9cc8-d9dee0206a75 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.443569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1348/2690] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:18:46.448010 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbffd9c-9d9d-4423-9bfa-e311253badbb admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.462960 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbffd9c-9d9d-4423-9bfa-e311253badbb admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.471278 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4fabc-9ecd-4ecb-bbea-b84c6c8d2c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.472954 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4fabc-9ecd-4ecb-bbea-b84c6c8d2c82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.473470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4fabc-9ecd-4ecb-bbea-b84c6c8d2c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.473993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4fabc-9ecd-4ecb-bbea-b84c6c8d2c82 admin admin] PATH_INFO: `/v3/projects/9b0b1b94221f442daa87ea1e3ca55df3/users/677282eb944e45d0ae92991660551c4e/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.503745 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4fabc-9ecd-4ecb-bbea-b84c6c8d2c82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b0b1b94221f442daa87ea1e3ca55df3, user_id=677282eb944e45d0ae92991660551c4e, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.509336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4fabc-9ecd-4ecb-bbea-b84c6c8d2c82 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.531767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1343/2691] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:46 2022] PUT /identity/v3/projects/9b0b1b94221f442daa87ea1e3ca55df3/users/677282eb944e45d0ae92991660551c4e/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:46.539147 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c96b05-cd9b-44c4-b186-7696540f3d31 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.539660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c96b05-cd9b-44c4-b186-7696540f3d31 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.540049 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c96b05-cd9b-44c4-b186-7696540f3d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.561096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-13c96b05-cd9b-44c4-b186-7696540f3d31 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:46.562524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-13c96b05-cd9b-44c4-b186-7696540f3d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:46.586832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:46.638177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1349/2692] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:46.651050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4fabc-9ecd-4ecb-bbea-b84c6c8d2c82 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.653777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-ced4fabc-9ecd-4ecb-bbea-b84c6c8d2c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:46.714587 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:46.739655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4fabc-9ecd-4ecb-bbea-b84c6c8d2c82 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.744206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700f6f5-d2f2-47cd-b609-2dc5834de6e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.744886 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700f6f5-d2f2-47cd-b609-2dc5834de6e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.745260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700f6f5-d2f2-47cd-b609-2dc5834de6e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.745429 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700f6f5-d2f2-47cd-b609-2dc5834de6e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.758259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700f6f5-d2f2-47cd-b609-2dc5834de6e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.759860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700f6f5-d2f2-47cd-b609-2dc5834de6e4 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.771527 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1344/2693] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:46.872832 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c96b05-cd9b-44c4-b186-7696540f3d31 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:46.874346 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-13c96b05-cd9b-44c4-b186-7696540f3d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:46.888634 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c96b05-cd9b-44c4-b186-7696540f3d31 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:46.894552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:46.895541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:46.895829 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:46.896104 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] PATH_INFO: `/v3/users/a96aa50932fe49438699b05ff3a45059` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:46.897621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a96aa50932fe49438699b05ff3a45059)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:46.899880 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:46.977901 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a96aa50932fe49438699b05ff3a45059'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:46.992094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a96aa50932fe49438699b05ff3a45059'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:46.993181 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a96aa50932fe49438699b05ff3a45059'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:47.010156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a96aa50932fe49438699b05ff3a45059'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:47.011256 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a96aa50932fe49438699b05ff3a45059'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:47.056001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94ac42-fa00-4d0c-b03a-25c98ad54f33 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:47.056395 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94ac42-fa00-4d0c-b03a-25c98ad54f33 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:47.056880 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94ac42-fa00-4d0c-b03a-25c98ad54f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:47.059586 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1350/2694] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:46 2022] DELETE /identity/v3/users/a96aa50932fe49438699b05ff3a45059 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:47.093779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-7e94ac42-fa00-4d0c-b03a-25c98ad54f33 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:47.095060 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7e94ac42-fa00-4d0c-b03a-25c98ad54f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:47.193252 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1345/2695] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:47.228573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:47.230693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:47.343698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34105a75-0f2a-4e4d-8527-7f311a030b60 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:47.351100 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd22db4-ae99-4ce7-8a82-4d84d9264be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:47.352041 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22db4-ae99-4ce7-8a82-4d84d9264be1 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:47.352373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22db4-ae99-4ce7-8a82-4d84d9264be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:47.352655 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22db4-ae99-4ce7-8a82-4d84d9264be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:47.363641 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd22db4-ae99-4ce7-8a82-4d84d9264be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:47.365123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd22db4-ae99-4ce7-8a82-4d84d9264be1 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:47.384992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1351/2696] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:47.971884 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94ac42-fa00-4d0c-b03a-25c98ad54f33 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:47.977231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7e94ac42-fa00-4d0c-b03a-25c98ad54f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:48.003283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94ac42-fa00-4d0c-b03a-25c98ad54f33 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:48.012927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:48.014784 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:48.015279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:48.015833 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] PATH_INFO: `/v3/projects/f42c151f821547968dd6ef65b8e9c3d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:48.018801 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f42c151f821547968dd6ef65b8e9c3d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:48.022716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:48.128569 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42c151f821547968dd6ef65b8e9c3d2', 'request_id': 'req-1996b84d-e6a1-4325-804f-f0df4b6ba142'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:48.137128 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42c151f821547968dd6ef65b8e9c3d2', 'request_id': 'req-1996b84d-e6a1-4325-804f-f0df4b6ba142'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:48.145232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42c151f821547968dd6ef65b8e9c3d2', 'request_id': 'req-1996b84d-e6a1-4325-804f-f0df4b6ba142'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:48.170167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] The token cache is being invalidate because project f42c151f821547968dd6ef65b8e9c3d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:48.171126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:48.173425 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:48.183238 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1346/2697] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:47 2022] DELETE /identity/v3/projects/f42c151f821547968dd6ef65b8e9c3d2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:48.185156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd22db4-ae99-4ce7-8a82-4d84d9264be1 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:48.204672 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd22db4-ae99-4ce7-8a82-4d84d9264be1 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:48.307156 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:48.348144 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317cf9ca-4b58-4ee1-be0c-21d35b5123c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:48.349544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317cf9ca-4b58-4ee1-be0c-21d35b5123c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:48.350044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317cf9ca-4b58-4ee1-be0c-21d35b5123c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:48.350519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317cf9ca-4b58-4ee1-be0c-21d35b5123c0 admin admin] PATH_INFO: `/v3/users/e00c1231d9df42bd8baff4e9fdbb52ed` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:48.353422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317cf9ca-4b58-4ee1-be0c-21d35b5123c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e00c1231d9df42bd8baff4e9fdbb52ed)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:48.357326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317cf9ca-4b58-4ee1-be0c-21d35b5123c0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:48.494012 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-317cf9ca-4b58-4ee1-be0c-21d35b5123c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e00c1231d9df42bd8baff4e9fdbb52ed'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:48.517363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-317cf9ca-4b58-4ee1-be0c-21d35b5123c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e00c1231d9df42bd8baff4e9fdbb52ed'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:48.524614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-317cf9ca-4b58-4ee1-be0c-21d35b5123c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e00c1231d9df42bd8baff4e9fdbb52ed'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:48.545408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-317cf9ca-4b58-4ee1-be0c-21d35b5123c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e00c1231d9df42bd8baff4e9fdbb52ed'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:48.546774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-317cf9ca-4b58-4ee1-be0c-21d35b5123c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e00c1231d9df42bd8baff4e9fdbb52ed'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:48.576939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1352/2698] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:48 2022] DELETE /identity/v3/users/e00c1231d9df42bd8baff4e9fdbb52ed => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:49.302929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:49.304926 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:49.327501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996b84d-e6a1-4325-804f-f0df4b6ba142 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:49.404327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:49.405233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:49.405520 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:49.405799 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b admin admin] PATH_INFO: `/v3/projects/4018179b5a6f467ba624542abbf47716` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:49.407127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4018179b5a6f467ba624542abbf47716)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:49.409725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:49.422823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:49.488955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4018179b5a6f467ba624542abbf47716', 'request_id': 'req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:49.496968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4018179b5a6f467ba624542abbf47716', 'request_id': 'req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:49.505986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4018179b5a6f467ba624542abbf47716', 'request_id': 'req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:49.526772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b admin admin] The token cache is being invalidate because project 4018179b5a6f467ba624542abbf47716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:49.527125 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:49.527806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-38dcdb55-0930-4fe2-be5a-8b341d23ff4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:49.529495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1347/2699] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:49 2022] DELETE /identity/v3/projects/4018179b5a6f467ba624542abbf47716 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:49.535090 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de39d3-f342-4ad5-b04f-cd3a8cb8fabe None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:49.536040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de39d3-f342-4ad5-b04f-cd3a8cb8fabe None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:49.536040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de39d3-f342-4ad5-b04f-cd3a8cb8fabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:49.560126 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-c7de39d3-f342-4ad5-b04f-cd3a8cb8fabe None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:49.561711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c7de39d3-f342-4ad5-b04f-cd3a8cb8fabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:49.648864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1353/2700] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:49.661958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742f2f9-3c61-4e53-9843-a03d09181548 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:49.662492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742f2f9-3c61-4e53-9843-a03d09181548 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:49.662908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742f2f9-3c61-4e53-9843-a03d09181548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:49.702710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c742f2f9-3c61-4e53-9843-a03d09181548 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:49.704301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c742f2f9-3c61-4e53-9843-a03d09181548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:49.715543 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1348/2701] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:49.720390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de39d3-f342-4ad5-b04f-cd3a8cb8fabe None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:49.745377 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de39d3-f342-4ad5-b04f-cd3a8cb8fabe None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:49.755633 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adee538-bde7-4b70-8985-1b5de22ca301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:49.756954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adee538-bde7-4b70-8985-1b5de22ca301 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:49.757436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adee538-bde7-4b70-8985-1b5de22ca301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:49.757913 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adee538-bde7-4b70-8985-1b5de22ca301 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:49.759260 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adee538-bde7-4b70-8985-1b5de22ca301 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:49.759828 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adee538-bde7-4b70-8985-1b5de22ca301 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:49.762986 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adee538-bde7-4b70-8985-1b5de22ca301 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:49.779883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1354/2702] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:18:49.787213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742f2f9-3c61-4e53-9843-a03d09181548 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:49.805818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742f2f9-3c61-4e53-9843-a03d09181548 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:49.813685 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66fa42-b963-449f-931c-860f34def940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:49.816292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66fa42-b963-449f-931c-860f34def940 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:49.816706 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66fa42-b963-449f-931c-860f34def940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:49.817135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66fa42-b963-449f-931c-860f34def940 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:49.817949 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66fa42-b963-449f-931c-860f34def940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:49.820276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66fa42-b963-449f-931c-860f34def940 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:49.863633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1349/2703] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:49 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:49.867936 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adee538-bde7-4b70-8985-1b5de22ca301 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:49.882848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adee538-bde7-4b70-8985-1b5de22ca301 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:49.975288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:50.018692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb95d3-bc3c-49fd-8b66-17a89a882bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:50.020510 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb95d3-bc3c-49fd-8b66-17a89a882bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:50.021034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb95d3-bc3c-49fd-8b66-17a89a882bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:50.021531 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb95d3-bc3c-49fd-8b66-17a89a882bdf admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:50.022888 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb95d3-bc3c-49fd-8b66-17a89a882bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:50.027354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb95d3-bc3c-49fd-8b66-17a89a882bdf admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:50.061750 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1355/2704] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:49 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:50.066491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66fa42-b963-449f-931c-860f34def940 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:50.079023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66fa42-b963-449f-931c-860f34def940 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:50.086077 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3e37f-fe23-4f6b-8d60-f63f1324e396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:50.086902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3e37f-fe23-4f6b-8d60-f63f1324e396 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:50.087302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3e37f-fe23-4f6b-8d60-f63f1324e396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:50.087662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3e37f-fe23-4f6b-8d60-f63f1324e396 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:50.088442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3e37f-fe23-4f6b-8d60-f63f1324e396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:50.090603 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3e37f-fe23-4f6b-8d60-f63f1324e396 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:50.105725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1350/2705] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:50.110482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb95d3-bc3c-49fd-8b66-17a89a882bdf admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:50.124537 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb95d3-bc3c-49fd-8b66-17a89a882bdf admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:50.134005 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd84d-eb97-4fc9-935f-448f071ee502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:50.135062 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd84d-eb97-4fc9-935f-448f071ee502 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:50.135880 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd84d-eb97-4fc9-935f-448f071ee502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:50.136343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd84d-eb97-4fc9-935f-448f071ee502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:50.137948 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fd84d-eb97-4fc9-935f-448f071ee502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:50.140641 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fd84d-eb97-4fc9-935f-448f071ee502 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:50.163495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1356/2706] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:50.168005 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3e37f-fe23-4f6b-8d60-f63f1324e396 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:50.183601 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3e37f-fe23-4f6b-8d60-f63f1324e396 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:50.205494 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61750ec-325f-4d69-a8d7-5c9d67dc42cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:50.212644 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61750ec-325f-4d69-a8d7-5c9d67dc42cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:50.213941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61750ec-325f-4d69-a8d7-5c9d67dc42cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:50.214470 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61750ec-325f-4d69-a8d7-5c9d67dc42cd admin admin] PATH_INFO: `/v3/projects/200b04f7dd164a258b3c8d13541bd149/users/db89242cefd749b489c6ec2eeaf7d845/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:50.259955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61750ec-325f-4d69-a8d7-5c9d67dc42cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=200b04f7dd164a258b3c8d13541bd149, user_id=db89242cefd749b489c6ec2eeaf7d845, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:50.263817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61750ec-325f-4d69-a8d7-5c9d67dc42cd admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:50.300035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1351/2707] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:50 2022] PUT /identity/v3/projects/200b04f7dd164a258b3c8d13541bd149/users/db89242cefd749b489c6ec2eeaf7d845/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:18:50.302418 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7048e55-2629-45f1-b614-45e97db5feaf None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:50.302715 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7048e55-2629-45f1-b614-45e97db5feaf None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:50.302997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7048e55-2629-45f1-b614-45e97db5feaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:50.338484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-a7048e55-2629-45f1-b614-45e97db5feaf None None] MFA Rules not processed for user `db89242cefd749b489c6ec2eeaf7d845`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:50.340556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a7048e55-2629-45f1-b614-45e97db5feaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:50.457224 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1357/2708] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:50.481170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61750ec-325f-4d69-a8d7-5c9d67dc42cd admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:50.484986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e61750ec-325f-4d69-a8d7-5c9d67dc42cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:50.669546 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61750ec-325f-4d69-a8d7-5c9d67dc42cd admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:50.685224 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cb5e6-f2fd-4172-be41-6b350a53959b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:50.686200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cb5e6-f2fd-4172-be41-6b350a53959b service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:50.686673 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cb5e6-f2fd-4172-be41-6b350a53959b service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:50.687171 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cb5e6-f2fd-4172-be41-6b350a53959b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:50.722211 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95cb5e6-f2fd-4172-be41-6b350a53959b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:50.725112 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95cb5e6-f2fd-4172-be41-6b350a53959b service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:50.745999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1352/2709] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:18:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:18:53.328552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cd413-6b53-4cd2-9b1f-2ddb00ec4fa3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:53.329119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cd413-6b53-4cd2-9b1f-2ddb00ec4fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:53.329407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5cd413-6b53-4cd2-9b1f-2ddb00ec4fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:53.370699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-2a5cd413-6b53-4cd2-9b1f-2ddb00ec4fa3 None None] MFA Rules not processed for user `677282eb944e45d0ae92991660551c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:53.372376 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2a5cd413-6b53-4cd2-9b1f-2ddb00ec4fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:53.475874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1358/2710] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:53.477203 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cb5e6-f2fd-4172-be41-6b350a53959b service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:53.478727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c95cb5e6-f2fd-4172-be41-6b350a53959b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:53.549840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:53.619995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cb5e6-f2fd-4172-be41-6b350a53959b service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:53.627850 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1669e7-e1fa-437a-90fd-2e0455c3288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:53.628553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1669e7-e1fa-437a-90fd-2e0455c3288c admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:53.628843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1669e7-e1fa-437a-90fd-2e0455c3288c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:53.629126 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1669e7-e1fa-437a-90fd-2e0455c3288c admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:53.630029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1669e7-e1fa-437a-90fd-2e0455c3288c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:53.632063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1669e7-e1fa-437a-90fd-2e0455c3288c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:53.675490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1353/2711] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:53 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:53.676723 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5cd413-6b53-4cd2-9b1f-2ddb00ec4fa3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:53.690848 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5cd413-6b53-4cd2-9b1f-2ddb00ec4fa3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:53.754416 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:53.775240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11085fb-dd65-4e22-8b56-0c6abf96a75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:53.775977 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11085fb-dd65-4e22-8b56-0c6abf96a75b admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:53.776268 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11085fb-dd65-4e22-8b56-0c6abf96a75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:53.776549 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11085fb-dd65-4e22-8b56-0c6abf96a75b admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:53.777660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11085fb-dd65-4e22-8b56-0c6abf96a75b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:53.780203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11085fb-dd65-4e22-8b56-0c6abf96a75b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:53.810989 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1359/2712] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:53 2022] POST /identity/v3/users => generated 299 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:53.817442 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1669e7-e1fa-437a-90fd-2e0455c3288c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:53.832127 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1669e7-e1fa-437a-90fd-2e0455c3288c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:53.839589 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880456b5-204b-406a-bc96-eaf33df99c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:53.840980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880456b5-204b-406a-bc96-eaf33df99c98 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:53.841275 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880456b5-204b-406a-bc96-eaf33df99c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:53.841562 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880456b5-204b-406a-bc96-eaf33df99c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:53.842923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880456b5-204b-406a-bc96-eaf33df99c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:53.844860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880456b5-204b-406a-bc96-eaf33df99c98 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:53.860773 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1354/2713] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:53.867188 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11085fb-dd65-4e22-8b56-0c6abf96a75b admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:53.882959 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11085fb-dd65-4e22-8b56-0c6abf96a75b admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:53.889367 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2dffc4-2a71-44da-a900-2b4bcef24367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:53.890313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dffc4-2a71-44da-a900-2b4bcef24367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:53.890597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dffc4-2a71-44da-a900-2b4bcef24367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:53.890895 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dffc4-2a71-44da-a900-2b4bcef24367 admin admin] PATH_INFO: `/v3/projects/c8e814c52ec14d749384e4793e146534/users/feb53ffd683245b7beb4034ba2b260a7/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:53.930775 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2dffc4-2a71-44da-a900-2b4bcef24367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e814c52ec14d749384e4793e146534, user_id=feb53ffd683245b7beb4034ba2b260a7, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:53.939541 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2dffc4-2a71-44da-a900-2b4bcef24367 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:53.967740 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1360/2714] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:53 2022] PUT /identity/v3/projects/c8e814c52ec14d749384e4793e146534/users/feb53ffd683245b7beb4034ba2b260a7/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:53.976382 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880456b5-204b-406a-bc96-eaf33df99c98 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:54.004907 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880456b5-204b-406a-bc96-eaf33df99c98 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:54.118379 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0a292-ed17-4668-99b4-826490cd79d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:54.119614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0a292-ed17-4668-99b4-826490cd79d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:54.120138 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0a292-ed17-4668-99b4-826490cd79d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:54.120638 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0a292-ed17-4668-99b4-826490cd79d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:54.122043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0a292-ed17-4668-99b4-826490cd79d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:54.125155 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0a292-ed17-4668-99b4-826490cd79d6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:54.146670 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1355/2715] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:54.150777 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2dffc4-2a71-44da-a900-2b4bcef24367 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:54.161997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2dffc4-2a71-44da-a900-2b4bcef24367 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:54.167028 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70691b46-700e-44b9-ac65-712aa572cbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:54.168111 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70691b46-700e-44b9-ac65-712aa572cbaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:54.168459 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70691b46-700e-44b9-ac65-712aa572cbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:54.168803 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70691b46-700e-44b9-ac65-712aa572cbaf admin admin] PATH_INFO: `/v3/projects/c8e814c52ec14d749384e4793e146534/users/feb53ffd683245b7beb4034ba2b260a7/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:54.171038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70691b46-700e-44b9-ac65-712aa572cbaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e814c52ec14d749384e4793e146534, user_id=feb53ffd683245b7beb4034ba2b260a7, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:54.174500 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70691b46-700e-44b9-ac65-712aa572cbaf admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:54.187154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1361/2716] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:54 2022] PUT /identity/v3/projects/c8e814c52ec14d749384e4793e146534/users/feb53ffd683245b7beb4034ba2b260a7/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:54.818040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0a292-ed17-4668-99b4-826490cd79d6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:54.821761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-33e0a292-ed17-4668-99b4-826490cd79d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:54.896215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0a292-ed17-4668-99b4-826490cd79d6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:55.001115 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:55.002463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:55.003762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:55.004214 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] PATH_INFO: `/v3/users/49c303616d64446dba3a9fab217a051e` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:55.005804 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c303616d64446dba3a9fab217a051e)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:55.010283 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:55.116436 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c303616d64446dba3a9fab217a051e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:55.117523 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c303616d64446dba3a9fab217a051e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:55.119813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:55.137590 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c303616d64446dba3a9fab217a051e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:55.159240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c303616d64446dba3a9fab217a051e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:55.163726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c303616d64446dba3a9fab217a051e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:55.203703 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1356/2717] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:54 2022] DELETE /identity/v3/users/49c303616d64446dba3a9fab217a051e => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:56.041325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70691b46-700e-44b9-ac65-712aa572cbaf admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:56.049745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-70691b46-700e-44b9-ac65-712aa572cbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:56.097784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70691b46-700e-44b9-ac65-712aa572cbaf admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:56.203754 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:56.206647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:56.207342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:56.208103 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] PATH_INFO: `/v3/projects/94432f6569274238a86b9695977c30db` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:56.211718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=94432f6569274238a86b9695977c30db)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:56.216906 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:56.347291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94432f6569274238a86b9695977c30db', 'request_id': 'req-90fbba7a-055e-498e-bb63-b8af9e6c55ed'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.362736 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94432f6569274238a86b9695977c30db', 'request_id': 'req-90fbba7a-055e-498e-bb63-b8af9e6c55ed'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.376809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94432f6569274238a86b9695977c30db', 'request_id': 'req-90fbba7a-055e-498e-bb63-b8af9e6c55ed'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.402208 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] The token cache is being invalidate because project 94432f6569274238a86b9695977c30db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:56.403082 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.404171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.407517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1362/2718] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:56 2022] DELETE /identity/v3/projects/94432f6569274238a86b9695977c30db => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:56.411890 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:56.424259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde033de-8415-49eb-a8fd-18aca6df4034 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:56.532665 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17fe78-221e-46a3-9592-e71edd04104c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:56.533931 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17fe78-221e-46a3-9592-e71edd04104c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:56.534515 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17fe78-221e-46a3-9592-e71edd04104c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:56.535152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17fe78-221e-46a3-9592-e71edd04104c admin admin] PATH_INFO: `/v3/users/d5f1382ca00c491092affa569b0b9a3f` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:56.538212 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17fe78-221e-46a3-9592-e71edd04104c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5f1382ca00c491092affa569b0b9a3f)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:56.542794 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17fe78-221e-46a3-9592-e71edd04104c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:56.650439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ec17fe78-221e-46a3-9592-e71edd04104c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5f1382ca00c491092affa569b0b9a3f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.651336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ec17fe78-221e-46a3-9592-e71edd04104c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5f1382ca00c491092affa569b0b9a3f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.663880 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ec17fe78-221e-46a3-9592-e71edd04104c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5f1382ca00c491092affa569b0b9a3f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.675428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ec17fe78-221e-46a3-9592-e71edd04104c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5f1382ca00c491092affa569b0b9a3f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.676305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-ec17fe78-221e-46a3-9592-e71edd04104c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5f1382ca00c491092affa569b0b9a3f'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:56.716483 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1357/2719] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:18:56 2022] DELETE /identity/v3/users/d5f1382ca00c491092affa569b0b9a3f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:57.198388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:57.203060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:57.251006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbba7a-055e-498e-bb63-b8af9e6c55ed admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:57.324271 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:57.432764 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:57.445358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:57.445862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:57.446375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc admin admin] PATH_INFO: `/v3/projects/b02ce14ecf394a15afed2dbd3616600e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:57.460949 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b02ce14ecf394a15afed2dbd3616600e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:57.464892 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:57.600784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b02ce14ecf394a15afed2dbd3616600e', 'request_id': 'req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:57.638501 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b02ce14ecf394a15afed2dbd3616600e', 'request_id': 'req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:57.679170 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b02ce14ecf394a15afed2dbd3616600e', 'request_id': 'req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:57.780325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc admin admin] The token cache is being invalidate because project b02ce14ecf394a15afed2dbd3616600e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:18:57.780325 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:57.780835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f5ec9006-9e0e-43f5-8103-3ed6f85fafdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:18:57.788950 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1363/2720] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:18:57 2022] DELETE /identity/v3/projects/b02ce14ecf394a15afed2dbd3616600e => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:57.791097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f165f15-c7aa-4e5d-a117-8df6a3245b4e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:57.792435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f165f15-c7aa-4e5d-a117-8df6a3245b4e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:57.792435 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f165f15-c7aa-4e5d-a117-8df6a3245b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:57.832850 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-5f165f15-c7aa-4e5d-a117-8df6a3245b4e None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:57.834380 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-5f165f15-c7aa-4e5d-a117-8df6a3245b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:57.937040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:57.994310 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1358/2721] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:57.996341 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e4364-9877-464d-a87f-a0eebcdde2ce None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:57.996638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e4364-9877-464d-a87f-a0eebcdde2ce None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:57.997513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e4364-9877-464d-a87f-a0eebcdde2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.034965 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-228e4364-9877-464d-a87f-a0eebcdde2ce None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:58.036358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-228e4364-9877-464d-a87f-a0eebcdde2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:58.052241 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1364/2722] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:58.053598 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f165f15-c7aa-4e5d-a117-8df6a3245b4e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:58.070571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f165f15-c7aa-4e5d-a117-8df6a3245b4e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:58.099491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e8740-8518-41db-882a-4b63a29a9f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:58.100570 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e8740-8518-41db-882a-4b63a29a9f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:58.100863 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e8740-8518-41db-882a-4b63a29a9f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:58.103787 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e8740-8518-41db-882a-4b63a29a9f60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.110202 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e8740-8518-41db-882a-4b63a29a9f60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:18:58.110702 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e8740-8518-41db-882a-4b63a29a9f60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:58.118071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e8740-8518-41db-882a-4b63a29a9f60 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:58.161061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1359/2723] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:18:58.166539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e4364-9877-464d-a87f-a0eebcdde2ce None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:58.205745 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e4364-9877-464d-a87f-a0eebcdde2ce None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:58.222478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba068d5-b7e9-4992-b78a-7d42ffe71f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:58.223841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba068d5-b7e9-4992-b78a-7d42ffe71f86 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:58.223841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba068d5-b7e9-4992-b78a-7d42ffe71f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:58.223841 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba068d5-b7e9-4992-b78a-7d42ffe71f86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.224494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba068d5-b7e9-4992-b78a-7d42ffe71f86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:58.226483 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba068d5-b7e9-4992-b78a-7d42ffe71f86 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:58.293103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e8740-8518-41db-882a-4b63a29a9f60 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:58.294408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1365/2724] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:58 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:58.309984 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e8740-8518-41db-882a-4b63a29a9f60 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:58.416272 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94bfcb-661d-465e-a883-f9028acae195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:58.417158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94bfcb-661d-465e-a883-f9028acae195 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:58.417838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94bfcb-661d-465e-a883-f9028acae195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:58.418789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94bfcb-661d-465e-a883-f9028acae195 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.419977 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94bfcb-661d-465e-a883-f9028acae195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:58.423436 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94bfcb-661d-465e-a883-f9028acae195 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:58.469382 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1360/2725] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:58 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:18:58.473699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba068d5-b7e9-4992-b78a-7d42ffe71f86 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:58.487710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba068d5-b7e9-4992-b78a-7d42ffe71f86 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:58.493929 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa22989-ecfd-4a8d-be00-47d3bf36977f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:58.494931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa22989-ecfd-4a8d-be00-47d3bf36977f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:58.495462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa22989-ecfd-4a8d-be00-47d3bf36977f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:58.495937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa22989-ecfd-4a8d-be00-47d3bf36977f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.496887 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa22989-ecfd-4a8d-be00-47d3bf36977f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:58.500240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa22989-ecfd-4a8d-be00-47d3bf36977f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:58.518854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1366/2726] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:58.520200 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94bfcb-661d-465e-a883-f9028acae195 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:58.543781 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94bfcb-661d-465e-a883-f9028acae195 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:58.553817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e293de-23a0-4c78-8b80-480d817fe59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:58.555136 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e293de-23a0-4c78-8b80-480d817fe59e admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:58.556437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e293de-23a0-4c78-8b80-480d817fe59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:58.560280 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e293de-23a0-4c78-8b80-480d817fe59e admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.560906 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e293de-23a0-4c78-8b80-480d817fe59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:58.563557 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e293de-23a0-4c78-8b80-480d817fe59e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:58.605123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1361/2727] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:58.610578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa22989-ecfd-4a8d-be00-47d3bf36977f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:58.632196 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa22989-ecfd-4a8d-be00-47d3bf36977f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:58.654455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e1d3a-dd8f-45fa-bd43-2b47d60862c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:58.655422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e1d3a-dd8f-45fa-bd43-2b47d60862c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:58.655720 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e1d3a-dd8f-45fa-bd43-2b47d60862c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:58.656002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e1d3a-dd8f-45fa-bd43-2b47d60862c6 admin admin] PATH_INFO: `/v3/projects/1c41d6c14be4481597e52e548acfe67c/users/fdd6494b4b044894a6d14a8c50625082/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.707858 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e1d3a-dd8f-45fa-bd43-2b47d60862c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c41d6c14be4481597e52e548acfe67c, user_id=fdd6494b4b044894a6d14a8c50625082, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:58.710578 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e1d3a-dd8f-45fa-bd43-2b47d60862c6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:58.738398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1367/2728] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:58 2022] PUT /identity/v3/projects/1c41d6c14be4481597e52e548acfe67c/users/fdd6494b4b044894a6d14a8c50625082/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:58.750183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1ad80-84ea-4201-b69e-f979c72e8ebe None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:58.750771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1ad80-84ea-4201-b69e-f979c72e8ebe None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:58.751212 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1ad80-84ea-4201-b69e-f979c72e8ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:58.850284 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-9ca1ad80-84ea-4201-b69e-f979c72e8ebe None None] MFA Rules not processed for user `fdd6494b4b044894a6d14a8c50625082`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:18:58.853788 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-9ca1ad80-84ea-4201-b69e-f979c72e8ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:18:58.985966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e1d3a-dd8f-45fa-bd43-2b47d60862c6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:58.989760 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1362/2729] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:18:59.100098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:59.107673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e1d3a-dd8f-45fa-bd43-2b47d60862c6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:59.115460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511654c-ce4f-447d-9d2f-1315c302981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:59.116235 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511654c-ce4f-447d-9d2f-1315c302981f admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:59.116623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511654c-ce4f-447d-9d2f-1315c302981f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:59.116977 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511654c-ce4f-447d-9d2f-1315c302981f admin admin] PATH_INFO: `/v3/projects` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:59.117893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511654c-ce4f-447d-9d2f-1315c302981f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:59.120769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511654c-ce4f-447d-9d2f-1315c302981f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:59.168282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1368/2730] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:18:58 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:59.172565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1ad80-84ea-4201-b69e-f979c72e8ebe None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:59.191779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1ad80-84ea-4201-b69e-f979c72e8ebe None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:59.285366 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:18:59.327866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e8452-38ff-44a1-8016-12b6254559fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:59.328580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e8452-38ff-44a1-8016-12b6254559fc admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:59.328867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e8452-38ff-44a1-8016-12b6254559fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:59.329145 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e8452-38ff-44a1-8016-12b6254559fc admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:59.329893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e8452-38ff-44a1-8016-12b6254559fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:59.331902 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e8452-38ff-44a1-8016-12b6254559fc admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:59.433767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1363/2731] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:18:59 2022] POST /identity/v3/users => generated 298 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:18:59.438105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511654c-ce4f-447d-9d2f-1315c302981f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:59.456172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511654c-ce4f-447d-9d2f-1315c302981f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:59.468209 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed53593-5a08-4561-9865-b9ca06894c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:59.471810 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed53593-5a08-4561-9865-b9ca06894c36 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:59.472852 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed53593-5a08-4561-9865-b9ca06894c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:59.473822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed53593-5a08-4561-9865-b9ca06894c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:59.477180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed53593-5a08-4561-9865-b9ca06894c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:59.482684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed53593-5a08-4561-9865-b9ca06894c36 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:59.521193 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1369/2732] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:59.525011 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e8452-38ff-44a1-8016-12b6254559fc admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:59.571705 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e8452-38ff-44a1-8016-12b6254559fc admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:59.577636 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e57875-97e4-4377-b8cb-e43e39d08558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:59.579291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e57875-97e4-4377-b8cb-e43e39d08558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:59.579951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e57875-97e4-4377-b8cb-e43e39d08558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:59.581044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e57875-97e4-4377-b8cb-e43e39d08558 admin admin] PATH_INFO: `/v3/projects/684a6e461e534e76b82eeb9918c24e36/users/508eab1f712345309ecabbbded795109/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:59.614475 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e57875-97e4-4377-b8cb-e43e39d08558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=684a6e461e534e76b82eeb9918c24e36, user_id=508eab1f712345309ecabbbded795109, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:59.623700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e57875-97e4-4377-b8cb-e43e39d08558 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:59.661417 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1364/2733] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:59 2022] PUT /identity/v3/projects/684a6e461e534e76b82eeb9918c24e36/users/508eab1f712345309ecabbbded795109/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:59.663020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed53593-5a08-4561-9865-b9ca06894c36 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:59.691864 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed53593-5a08-4561-9865-b9ca06894c36 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:59.843163 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e14031-91fe-479f-9729-e8520952e91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:59.844956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e14031-91fe-479f-9729-e8520952e91e admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:59.845430 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e14031-91fe-479f-9729-e8520952e91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:59.846442 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e14031-91fe-479f-9729-e8520952e91e admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:59.847452 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e14031-91fe-479f-9729-e8520952e91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:59.849467 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e14031-91fe-479f-9729-e8520952e91e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:59.870822 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1370/2734] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:18:59.875903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e57875-97e4-4377-b8cb-e43e39d08558 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:18:59.898795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e57875-97e4-4377-b8cb-e43e39d08558 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:18:59.906299 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b666de5-bb9b-49d6-8af3-d8a092f57ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:18:59.907744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b666de5-bb9b-49d6-8af3-d8a092f57ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:59.908873 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b666de5-bb9b-49d6-8af3-d8a092f57ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:59.909528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b666de5-bb9b-49d6-8af3-d8a092f57ed9 admin admin] PATH_INFO: `/v3/projects/684a6e461e534e76b82eeb9918c24e36/users/508eab1f712345309ecabbbded795109/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:59.913358 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b666de5-bb9b-49d6-8af3-d8a092f57ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=684a6e461e534e76b82eeb9918c24e36, user_id=508eab1f712345309ecabbbded795109, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:18:59.916726 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b666de5-bb9b-49d6-8af3-d8a092f57ed9 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:18:59.942833 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1365/2735] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:18:59 2022] PUT /identity/v3/projects/684a6e461e534e76b82eeb9918c24e36/users/508eab1f712345309ecabbbded795109/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:18:59.947232 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa18229-cfe4-494d-a9f3-9138c553f757 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:18:59.947845 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa18229-cfe4-494d-a9f3-9138c553f757 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:18:59.948315 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa18229-cfe4-494d-a9f3-9138c553f757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:18:59.998429 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-5fa18229-cfe4-494d-a9f3-9138c553f757 None None] MFA Rules not processed for user `508eab1f712345309ecabbbded795109`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:00.000575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5fa18229-cfe4-494d-a9f3-9138c553f757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:00.176343 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1371/2736] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:18:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:00.196222 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b666de5-bb9b-49d6-8af3-d8a092f57ed9 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:00.200146 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0b666de5-bb9b-49d6-8af3-d8a092f57ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:00.381335 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b666de5-bb9b-49d6-8af3-d8a092f57ed9 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:00.393783 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1727a5f-aa48-4cd5-aa5f-000d5e3c48fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:00.394484 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1727a5f-aa48-4cd5-aa5f-000d5e3c48fd service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:00.394779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1727a5f-aa48-4cd5-aa5f-000d5e3c48fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:00.395066 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1727a5f-aa48-4cd5-aa5f-000d5e3c48fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:00.497899 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1727a5f-aa48-4cd5-aa5f-000d5e3c48fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:00.499598 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1727a5f-aa48-4cd5-aa5f-000d5e3c48fd service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:00.513923 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1366/2737] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:01.178067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa18229-cfe4-494d-a9f3-9138c553f757 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:01.186454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5fa18229-cfe4-494d-a9f3-9138c553f757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:01.232276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e754f-32c1-43df-8b29-f7cb36128365 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:01.233036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e754f-32c1-43df-8b29-f7cb36128365 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:01.233507 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e754f-32c1-43df-8b29-f7cb36128365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:01.340186 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-580e754f-32c1-43df-8b29-f7cb36128365 None None] MFA Rules not processed for user `feb53ffd683245b7beb4034ba2b260a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:01.349411 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-580e754f-32c1-43df-8b29-f7cb36128365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:01.359123 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:01.420808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa18229-cfe4-494d-a9f3-9138c553f757 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:01.422045 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:01.435146 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f5a7d-d740-4b2b-9423-0305798dc3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:01.436283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f5a7d-d740-4b2b-9423-0305798dc3f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:01.436684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f5a7d-d740-4b2b-9423-0305798dc3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:01.437064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f5a7d-d740-4b2b-9423-0305798dc3f5 admin admin] PATH_INFO: `/v3/users/db89242cefd749b489c6ec2eeaf7d845` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:01.440354 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f5a7d-d740-4b2b-9423-0305798dc3f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db89242cefd749b489c6ec2eeaf7d845)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:01.442767 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f5a7d-d740-4b2b-9423-0305798dc3f5 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:01.540008 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1367/2738] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:01.551361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e754f-32c1-43df-8b29-f7cb36128365 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:01.553028 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-580e754f-32c1-43df-8b29-f7cb36128365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:01.568369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e11f5a7d-d740-4b2b-9423-0305798dc3f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db89242cefd749b489c6ec2eeaf7d845'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:01.586561 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e11f5a7d-d740-4b2b-9423-0305798dc3f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db89242cefd749b489c6ec2eeaf7d845'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:01.588019 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e11f5a7d-d740-4b2b-9423-0305798dc3f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db89242cefd749b489c6ec2eeaf7d845'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:01.602694 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e11f5a7d-d740-4b2b-9423-0305798dc3f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db89242cefd749b489c6ec2eeaf7d845'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:01.607120 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-e11f5a7d-d740-4b2b-9423-0305798dc3f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db89242cefd749b489c6ec2eeaf7d845'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:01.635579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1372/2739] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:19:01 2022] DELETE /identity/v3/users/db89242cefd749b489c6ec2eeaf7d845 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:01.640820 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39042fe3-82c9-422a-8544-9c373246ca13 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:01.641594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39042fe3-82c9-422a-8544-9c373246ca13 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:01.642110 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39042fe3-82c9-422a-8544-9c373246ca13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:01.668539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e754f-32c1-43df-8b29-f7cb36128365 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:01.672644 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-39042fe3-82c9-422a-8544-9c373246ca13 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:01.674282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-39042fe3-82c9-422a-8544-9c373246ca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:01.757071 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b46777-505a-4e0c-bd7f-6145acf8712d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1ad1ffe4d99420cb568d6154f779838', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:01.758393 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b46777-505a-4e0c-bd7f-6145acf8712d service glance] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:01.760396 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b46777-505a-4e0c-bd7f-6145acf8712d service glance] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:01.761077 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b46777-505a-4e0c-bd7f-6145acf8712d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:01.763727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-75b46777-505a-4e0c-bd7f-6145acf8712d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:01.798367 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1373/2740] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:19:01.811802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39042fe3-82c9-422a-8544-9c373246ca13 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:01.813266 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-39042fe3-82c9-422a-8544-9c373246ca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:01.883936 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b46777-505a-4e0c-bd7f-6145acf8712d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:01.886042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b46777-505a-4e0c-bd7f-6145acf8712d service glance] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:01.909719 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1368/2741] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:19:01.937215 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39042fe3-82c9-422a-8544-9c373246ca13 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:01.946670 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa31244-df99-43f3-8677-210fddd5eee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:01.950851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa31244-df99-43f3-8677-210fddd5eee8 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:01.951480 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa31244-df99-43f3-8677-210fddd5eee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:01.951992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa31244-df99-43f3-8677-210fddd5eee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:01.981471 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa31244-df99-43f3-8677-210fddd5eee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:01.982993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa31244-df99-43f3-8677-210fddd5eee8 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:02.003652 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1374/2742] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:19:02.061659 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78385836-84c2-44ea-9d5a-bdf10ef1cca1 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:02.062076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78385836-84c2-44ea-9d5a-bdf10ef1cca1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:02.062299 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78385836-84c2-44ea-9d5a-bdf10ef1cca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:02.109720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:02.158894 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-78385836-84c2-44ea-9d5a-bdf10ef1cca1 None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:02.161550 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-78385836-84c2-44ea-9d5a-bdf10ef1cca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:02.308454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1369/2743] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:19:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:02.380055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa31244-df99-43f3-8677-210fddd5eee8 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:02.382289 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5fa31244-df99-43f3-8677-210fddd5eee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:02.393677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78385836-84c2-44ea-9d5a-bdf10ef1cca1 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:02.393969 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-78385836-84c2-44ea-9d5a-bdf10ef1cca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:02.410079 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78385836-84c2-44ea-9d5a-bdf10ef1cca1 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:02.416133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177b78f-122c-4166-84a0-ac09af97b146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:02.417486 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177b78f-122c-4166-84a0-ac09af97b146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:02.418133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177b78f-122c-4166-84a0-ac09af97b146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:02.418522 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177b78f-122c-4166-84a0-ac09af97b146 admin admin] PATH_INFO: `/v3/projects/200b04f7dd164a258b3c8d13541bd149` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:02.420565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177b78f-122c-4166-84a0-ac09af97b146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=200b04f7dd164a258b3c8d13541bd149)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:02.422719 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177b78f-122c-4166-84a0-ac09af97b146 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:02.431072 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:02.542815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4177b78f-122c-4166-84a0-ac09af97b146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200b04f7dd164a258b3c8d13541bd149', 'request_id': 'req-4177b78f-122c-4166-84a0-ac09af97b146'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:02.559860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4177b78f-122c-4166-84a0-ac09af97b146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200b04f7dd164a258b3c8d13541bd149', 'request_id': 'req-4177b78f-122c-4166-84a0-ac09af97b146'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:02.570442 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa31244-df99-43f3-8677-210fddd5eee8 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:02.575181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4177b78f-122c-4166-84a0-ac09af97b146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200b04f7dd164a258b3c8d13541bd149', 'request_id': 'req-4177b78f-122c-4166-84a0-ac09af97b146'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:02.606623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4177b78f-122c-4166-84a0-ac09af97b146 admin admin] The token cache is being invalidate because project 200b04f7dd164a258b3c8d13541bd149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:19:02.607058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4177b78f-122c-4166-84a0-ac09af97b146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:02.608535 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4177b78f-122c-4166-84a0-ac09af97b146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:02.615121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1370/2744] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:19:02 2022] DELETE /identity/v3/projects/200b04f7dd164a258b3c8d13541bd149 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:19:02.617598 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ee5f1-383f-405b-9480-905042ed8c7b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:02.618324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ee5f1-383f-405b-9480-905042ed8c7b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:02.618712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ee5f1-383f-405b-9480-905042ed8c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:02.649521 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-fa2ee5f1-383f-405b-9480-905042ed8c7b None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:02.652011 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-fa2ee5f1-383f-405b-9480-905042ed8c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:02.685770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844774ae-14a7-418a-a540-21b1e7799397 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:02.686532 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844774ae-14a7-418a-a540-21b1e7799397 service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:02.686983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844774ae-14a7-418a-a540-21b1e7799397 service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:02.687265 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844774ae-14a7-418a-a540-21b1e7799397 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:02.689380 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-844774ae-14a7-418a-a540-21b1e7799397 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:02.786832 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1371/2745] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:02.789578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f397423-9a92-4235-ad33-e423891eff38 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:02.790672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f397423-9a92-4235-ad33-e423891eff38 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:02.790672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f397423-9a92-4235-ad33-e423891eff38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:02.821282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844774ae-14a7-418a-a540-21b1e7799397 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:02.823009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844774ae-14a7-418a-a540-21b1e7799397 service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:02.837469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1375/2746] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:19:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 11:19:02.851087 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-2f397423-9a92-4235-ad33-e423891eff38 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:02.853488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2f397423-9a92-4235-ad33-e423891eff38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:02.884501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1372/2747] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:02.890116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844774ae-14a7-418a-a540-21b1e7799397 service swift] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:02.907910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844774ae-14a7-418a-a540-21b1e7799397 service swift] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:02.931216 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1737949-cf77-49d8-9b85-bacb5198f2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:02.932051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1737949-cf77-49d8-9b85-bacb5198f2bb admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:02.932326 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1737949-cf77-49d8-9b85-bacb5198f2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:02.932666 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1737949-cf77-49d8-9b85-bacb5198f2bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:02.933444 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1737949-cf77-49d8-9b85-bacb5198f2bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:19:02.933853 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1737949-cf77-49d8-9b85-bacb5198f2bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:02.936323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1737949-cf77-49d8-9b85-bacb5198f2bb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:02.989305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f397423-9a92-4235-ad33-e423891eff38 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:02.989862 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1376/2748] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:19:03.014013 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f397423-9a92-4235-ad33-e423891eff38 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:03.038498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520999c8-9844-4498-b3b6-1a6c78cde9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:03.040013 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520999c8-9844-4498-b3b6-1a6c78cde9ed admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:03.040508 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520999c8-9844-4498-b3b6-1a6c78cde9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:03.041081 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520999c8-9844-4498-b3b6-1a6c78cde9ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:03.042466 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520999c8-9844-4498-b3b6-1a6c78cde9ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:03.046667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520999c8-9844-4498-b3b6-1a6c78cde9ed admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:03.068031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:03.151460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1737949-cf77-49d8-9b85-bacb5198f2bb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:03.153096 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1373/2749] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:19:02 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:19:03.175770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1737949-cf77-49d8-9b85-bacb5198f2bb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:03.226438 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=100 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:03.343871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4d080-068b-44a0-a4fc-0ebca5f7d5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:03.344978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4d080-068b-44a0-a4fc-0ebca5f7d5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:03.345722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4d080-068b-44a0-a4fc-0ebca5f7d5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:03.346000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4d080-068b-44a0-a4fc-0ebca5f7d5f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:03.347363 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4d080-068b-44a0-a4fc-0ebca5f7d5f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:03.351792 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4d080-068b-44a0-a4fc-0ebca5f7d5f6 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:03.397844 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1377/2750] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:19:03 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:19:03.404916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520999c8-9844-4498-b3b6-1a6c78cde9ed admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:03.425056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520999c8-9844-4498-b3b6-1a6c78cde9ed admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:03.431867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502399b-301a-45d6-a617-02c0aab48ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:03.435134 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502399b-301a-45d6-a617-02c0aab48ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:03.436840 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502399b-301a-45d6-a617-02c0aab48ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:03.437413 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502399b-301a-45d6-a617-02c0aab48ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:03.438447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502399b-301a-45d6-a617-02c0aab48ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:03.441640 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502399b-301a-45d6-a617-02c0aab48ca2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:03.472935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1374/2751] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:19:03.477675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4d080-068b-44a0-a4fc-0ebca5f7d5f6 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:03.494335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4d080-068b-44a0-a4fc-0ebca5f7d5f6 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:03.501472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8c65c-f129-49bb-9d64-af85b160e7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:03.503907 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8c65c-f129-49bb-9d64-af85b160e7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:03.504530 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8c65c-f129-49bb-9d64-af85b160e7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:03.505000 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8c65c-f129-49bb-9d64-af85b160e7e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:03.508138 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8c65c-f129-49bb-9d64-af85b160e7e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:03.510680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8c65c-f129-49bb-9d64-af85b160e7e3 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:03.528402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1378/2752] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:19:03.535774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502399b-301a-45d6-a617-02c0aab48ca2 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:03.548638 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502399b-301a-45d6-a617-02c0aab48ca2 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:03.557351 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0865a-7dc8-4490-8223-2c8d532f0e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:03.558583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0865a-7dc8-4490-8223-2c8d532f0e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:03.559041 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0865a-7dc8-4490-8223-2c8d532f0e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:03.559569 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0865a-7dc8-4490-8223-2c8d532f0e88 admin admin] PATH_INFO: `/v3/projects/92623b5296c4468c97f4c975a54d8495/users/38c6d95788a345ef8a05eff6098bb2c2/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:03.586023 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0865a-7dc8-4490-8223-2c8d532f0e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92623b5296c4468c97f4c975a54d8495, user_id=38c6d95788a345ef8a05eff6098bb2c2, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:03.589368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0865a-7dc8-4490-8223-2c8d532f0e88 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:03.616226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1375/2753] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:19:03 2022] PUT /identity/v3/projects/92623b5296c4468c97f4c975a54d8495/users/38c6d95788a345ef8a05eff6098bb2c2/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:03.692393 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476d27a-2689-475b-8d3e-a1e71e9de50f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:03.693109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476d27a-2689-475b-8d3e-a1e71e9de50f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:03.693613 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476d27a-2689-475b-8d3e-a1e71e9de50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:03.757498 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8476d27a-2689-475b-8d3e-a1e71e9de50f None None] MFA Rules not processed for user `38c6d95788a345ef8a05eff6098bb2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:03.759807 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8476d27a-2689-475b-8d3e-a1e71e9de50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:03.952187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1379/2754] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:03.967536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0865a-7dc8-4490-8223-2c8d532f0e88 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:03.969796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-e1c0865a-7dc8-4490-8223-2c8d532f0e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:04.068436 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0865a-7dc8-4490-8223-2c8d532f0e88 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:04.072813 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db3148-7658-4542-b955-89bf9b7d1414 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1ad1ffe4d99420cb568d6154f779838', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:04.074156 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db3148-7658-4542-b955-89bf9b7d1414 service glance] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:04.074676 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db3148-7658-4542-b955-89bf9b7d1414 service glance] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:04.075177 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db3148-7658-4542-b955-89bf9b7d1414 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:04.087141 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db3148-7658-4542-b955-89bf9b7d1414 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:04.089491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db3148-7658-4542-b955-89bf9b7d1414 service glance] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:04.107462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1376/2755] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:07.152557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476d27a-2689-475b-8d3e-a1e71e9de50f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:07.154878 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8476d27a-2689-475b-8d3e-a1e71e9de50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:07.307886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476d27a-2689-475b-8d3e-a1e71e9de50f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:07.317873 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fda3b6-3fe9-45e3-b712-56d938462b04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:07.318679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fda3b6-3fe9-45e3-b712-56d938462b04 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:07.318974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fda3b6-3fe9-45e3-b712-56d938462b04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:07.319255 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fda3b6-3fe9-45e3-b712-56d938462b04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:07.327999 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-64fda3b6-3fe9-45e3-b712-56d938462b04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:07.459537 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:07.540502 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fda3b6-3fe9-45e3-b712-56d938462b04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:07.543462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fda3b6-3fe9-45e3-b712-56d938462b04 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:07.563335 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1380/2756] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:19:13.288135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db3148-7658-4542-b955-89bf9b7d1414 service glance] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:13.290365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c7db3148-7658-4542-b955-89bf9b7d1414 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:13.459526 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db3148-7658-4542-b955-89bf9b7d1414 service glance] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:13.470980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:13.472122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:13.472542 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:13.472972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] PATH_INFO: `/v3/users/fdd6494b4b044894a6d14a8c50625082` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:13.474592 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdd6494b4b044894a6d14a8c50625082)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:13.478748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:13.574206 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdd6494b4b044894a6d14a8c50625082'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:13.574991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdd6494b4b044894a6d14a8c50625082'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:13.587289 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdd6494b4b044894a6d14a8c50625082'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:13.590946 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:13.598910 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdd6494b4b044894a6d14a8c50625082'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:13.599626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdd6494b4b044894a6d14a8c50625082'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:13.637948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1377/2757] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:19:13 2022] DELETE /identity/v3/users/fdd6494b4b044894a6d14a8c50625082 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:13.642939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e8f85-e16d-4d8a-81df-9a06458a1c45 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:13.644024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e8f85-e16d-4d8a-81df-9a06458a1c45 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:13.644024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e8f85-e16d-4d8a-81df-9a06458a1c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:13.675469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d11e8f85-e16d-4d8a-81df-9a06458a1c45 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:13.675469 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d11e8f85-e16d-4d8a-81df-9a06458a1c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:13.796716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1381/2758] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:13.808076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:13.810433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:13.933361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b17dc-fceb-43f1-85f3-5860bef57ed6 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:13.937431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cfc75-28de-443f-aedc-9045e40502be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:13.938251 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cfc75-28de-443f-aedc-9045e40502be service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:13.938635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cfc75-28de-443f-aedc-9045e40502be service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:13.939052 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cfc75-28de-443f-aedc-9045e40502be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:13.952591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cfc75-28de-443f-aedc-9045e40502be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:13.954228 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cfc75-28de-443f-aedc-9045e40502be service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:13.971767 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1378/2759] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:14.453718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e8f85-e16d-4d8a-81df-9a06458a1c45 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:14.456124 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d11e8f85-e16d-4d8a-81df-9a06458a1c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:14.479631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e8f85-e16d-4d8a-81df-9a06458a1c45 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:14.490569 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:14.491673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:14.492038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:14.492380 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4 admin admin] PATH_INFO: `/v3/projects/1c41d6c14be4481597e52e548acfe67c` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:14.493844 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c41d6c14be4481597e52e548acfe67c)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:14.495967 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:14.564997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c41d6c14be4481597e52e548acfe67c', 'request_id': 'req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:14.574575 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c41d6c14be4481597e52e548acfe67c', 'request_id': 'req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:14.592313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c41d6c14be4481597e52e548acfe67c', 'request_id': 'req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:14.607938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4 admin admin] The token cache is being invalidate because project 1c41d6c14be4481597e52e548acfe67c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:19:14.608234 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:14.608931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a2910ea8-fab9-4bce-90d0-20c7b635d9d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:14.610539 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1382/2760] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:19:14 2022] DELETE /identity/v3/projects/1c41d6c14be4481597e52e548acfe67c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:14.615932 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cfc75-28de-443f-aedc-9045e40502be service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:14.627872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cfc75-28de-443f-aedc-9045e40502be service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:14.733650 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:14.734970 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:14.735322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:14.735730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] PATH_INFO: `/v3/users/508eab1f712345309ecabbbded795109` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:14.737818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508eab1f712345309ecabbbded795109)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:14.740635 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:14.816604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508eab1f712345309ecabbbded795109'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:14.817463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508eab1f712345309ecabbbded795109'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:14.829510 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508eab1f712345309ecabbbded795109'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:14.843276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508eab1f712345309ecabbbded795109'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:14.844490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508eab1f712345309ecabbbded795109'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:14.847756 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7d04a-6d74-4e76-bfd9-10e2cf0f6ecb None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:14.848531 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7d04a-6d74-4e76-bfd9-10e2cf0f6ecb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:14.848985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7d04a-6d74-4e76-bfd9-10e2cf0f6ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:14.861733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:14.869681 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1379/2761] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:19:14 2022] DELETE /identity/v3/users/508eab1f712345309ecabbbded795109 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:14.879447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-e0b7d04a-6d74-4e76-bfd9-10e2cf0f6ecb None None] MFA Rules not processed for user `677282eb944e45d0ae92991660551c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:14.881439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-e0b7d04a-6d74-4e76-bfd9-10e2cf0f6ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:14.987439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1383/2762] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:19:15.294449 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:15.296165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:15.421948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85544a2e-4e64-4bab-87d3-aea64a2ccdd5 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:15.430586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:15.432025 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:15.432556 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:15.432962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe admin admin] PATH_INFO: `/v3/projects/684a6e461e534e76b82eeb9918c24e36` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:15.435195 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=684a6e461e534e76b82eeb9918c24e36)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:15.437355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:15.525401 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '684a6e461e534e76b82eeb9918c24e36', 'request_id': 'req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:15.534219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '684a6e461e534e76b82eeb9918c24e36', 'request_id': 'req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:15.548279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '684a6e461e534e76b82eeb9918c24e36', 'request_id': 'req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:15.576714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe admin admin] The token cache is being invalidate because project 684a6e461e534e76b82eeb9918c24e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:19:15.577463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:15.578498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-1e12db19-3970-4c9b-a7b9-8c4c21f06dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:15.581796 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1380/2763] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:19:15 2022] DELETE /identity/v3/projects/684a6e461e534e76b82eeb9918c24e36 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:19:15.590909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a99d6-1d72-4d1f-80bb-a30fdea719c5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:15.591414 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a99d6-1d72-4d1f-80bb-a30fdea719c5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:15.591932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a99d6-1d72-4d1f-80bb-a30fdea719c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:15.620660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4a0a99d6-1d72-4d1f-80bb-a30fdea719c5 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:15.622800 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4a0a99d6-1d72-4d1f-80bb-a30fdea719c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:15.633875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:15.717692 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1384/2764] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:15.733083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccd471-5783-4f29-ac4f-a86c0d2b0b51 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:15.733744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccd471-5783-4f29-ac4f-a86c0d2b0b51 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:15.733744 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccd471-5783-4f29-ac4f-a86c0d2b0b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:15.766841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-c3ccd471-5783-4f29-ac4f-a86c0d2b0b51 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:15.769720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c3ccd471-5783-4f29-ac4f-a86c0d2b0b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:15.780040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1381/2765] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:15.784804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a99d6-1d72-4d1f-80bb-a30fdea719c5 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:15.795930 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a99d6-1d72-4d1f-80bb-a30fdea719c5 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:15.801203 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227adb9-cd0b-4ef2-80b7-a71d55e5c255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:15.801903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227adb9-cd0b-4ef2-80b7-a71d55e5c255 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:15.802184 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227adb9-cd0b-4ef2-80b7-a71d55e5c255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:15.802462 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227adb9-cd0b-4ef2-80b7-a71d55e5c255 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:15.803245 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227adb9-cd0b-4ef2-80b7-a71d55e5c255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:19:15.803594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227adb9-cd0b-4ef2-80b7-a71d55e5c255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:15.805561 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227adb9-cd0b-4ef2-80b7-a71d55e5c255 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:15.821586 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1385/2766] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:19:15.827229 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccd471-5783-4f29-ac4f-a86c0d2b0b51 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:15.844621 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccd471-5783-4f29-ac4f-a86c0d2b0b51 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:15.853519 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320d2aa-ff79-4cb7-9cfb-d866fa4f9ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:15.855358 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320d2aa-ff79-4cb7-9cfb-d866fa4f9ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:15.855853 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320d2aa-ff79-4cb7-9cfb-d866fa4f9ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:15.856239 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320d2aa-ff79-4cb7-9cfb-d866fa4f9ad2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:15.857305 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320d2aa-ff79-4cb7-9cfb-d866fa4f9ad2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:15.860189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320d2aa-ff79-4cb7-9cfb-d866fa4f9ad2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:15.916234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1382/2767] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:19:15 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:19:15.921491 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227adb9-cd0b-4ef2-80b7-a71d55e5c255 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:15.927786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320d2aa-ff79-4cb7-9cfb-d866fa4f9ad2 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:15.930776 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6320d2aa-ff79-4cb7-9cfb-d866fa4f9ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:15.943518 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227adb9-cd0b-4ef2-80b7-a71d55e5c255 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:16.067555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86386472-bbc0-41ce-a2a3-c9be2eeab0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:16.067555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86386472-bbc0-41ce-a2a3-c9be2eeab0ea admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:16.067555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86386472-bbc0-41ce-a2a3-c9be2eeab0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:16.067555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86386472-bbc0-41ce-a2a3-c9be2eeab0ea admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:16.068631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86386472-bbc0-41ce-a2a3-c9be2eeab0ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:16.071312 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86386472-bbc0-41ce-a2a3-c9be2eeab0ea admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:16.076241 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:16.088361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320d2aa-ff79-4cb7-9cfb-d866fa4f9ad2 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:16.101777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3635fc3-d1a1-48fc-9eda-800d9bf42ea4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:16.102338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3635fc3-d1a1-48fc-9eda-800d9bf42ea4 service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:16.102623 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3635fc3-d1a1-48fc-9eda-800d9bf42ea4 service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:16.102842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1386/2768] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:19:15 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:19:16.102956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3635fc3-d1a1-48fc-9eda-800d9bf42ea4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:16.104907 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b3635fc3-d1a1-48fc-9eda-800d9bf42ea4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:16.108322 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86386472-bbc0-41ce-a2a3-c9be2eeab0ea admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:16.118606 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3635fc3-d1a1-48fc-9eda-800d9bf42ea4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:16.120299 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3635fc3-d1a1-48fc-9eda-800d9bf42ea4 service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:16.121796 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86386472-bbc0-41ce-a2a3-c9be2eeab0ea admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:16.128246 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce8cc-33ba-43ce-a29f-33b270a3796f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:16.129314 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce8cc-33ba-43ce-a29f-33b270a3796f admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:16.129768 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce8cc-33ba-43ce-a29f-33b270a3796f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:16.130189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce8cc-33ba-43ce-a29f-33b270a3796f admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:16.131429 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ce8cc-33ba-43ce-a29f-33b270a3796f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:16.134367 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ce8cc-33ba-43ce-a29f-33b270a3796f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:16.135865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1383/2769] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:16.148842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1387/2770] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:19:16.154612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3635fc3-d1a1-48fc-9eda-800d9bf42ea4 service placement] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:16.166708 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3635fc3-d1a1-48fc-9eda-800d9bf42ea4 service placement] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:16.172304 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36736e57-28eb-4556-9382-b1a9279f837b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:16.173193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36736e57-28eb-4556-9382-b1a9279f837b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:16.174218 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36736e57-28eb-4556-9382-b1a9279f837b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:16.174621 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36736e57-28eb-4556-9382-b1a9279f837b admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:16.175506 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36736e57-28eb-4556-9382-b1a9279f837b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:16.177371 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36736e57-28eb-4556-9382-b1a9279f837b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:16.194012 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1384/2771] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:19:16.200790 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce8cc-33ba-43ce-a29f-33b270a3796f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:16.217472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce8cc-33ba-43ce-a29f-33b270a3796f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:16.224439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b2e34-6aa2-4b5d-9876-f7b245a018b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:16.226101 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b2e34-6aa2-4b5d-9876-f7b245a018b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:16.226586 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b2e34-6aa2-4b5d-9876-f7b245a018b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:16.227074 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b2e34-6aa2-4b5d-9876-f7b245a018b4 admin admin] PATH_INFO: `/v3/projects/442d027d621e40e2800fd6467f2bb60d/users/831f85e5678248b6b2991b8ac3a5e2b6/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:16.263337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b2e34-6aa2-4b5d-9876-f7b245a018b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=442d027d621e40e2800fd6467f2bb60d, user_id=831f85e5678248b6b2991b8ac3a5e2b6, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:16.269952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b2e34-6aa2-4b5d-9876-f7b245a018b4 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:16.291329 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1388/2772] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:19:16 2022] PUT /identity/v3/projects/442d027d621e40e2800fd6467f2bb60d/users/831f85e5678248b6b2991b8ac3a5e2b6/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:16.300203 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56ba39-0422-4833-bb43-3aa0ab2a92a1 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:16.300723 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56ba39-0422-4833-bb43-3aa0ab2a92a1 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:16.301225 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56ba39-0422-4833-bb43-3aa0ab2a92a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:16.334881 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-bc56ba39-0422-4833-bb43-3aa0ab2a92a1 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:16.337226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bc56ba39-0422-4833-bb43-3aa0ab2a92a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:16.454792 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1385/2773] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:16.465058 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b2e34-6aa2-4b5d-9876-f7b245a018b4 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:16.466893 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d47b2e34-6aa2-4b5d-9876-f7b245a018b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:16.567526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b2e34-6aa2-4b5d-9876-f7b245a018b4 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:16.575562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afef36b-969d-4214-b379-0bef87b481a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:16.576312 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afef36b-969d-4214-b379-0bef87b481a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:16.576619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afef36b-969d-4214-b379-0bef87b481a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:16.576904 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afef36b-969d-4214-b379-0bef87b481a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:16.586484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:16.589630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afef36b-969d-4214-b379-0bef87b481a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:16.591025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afef36b-969d-4214-b379-0bef87b481a7 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:16.608069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1389/2774] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:19:17.656287 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23330d31-5d1a-4f0c-a27f-3ebaa3d91035 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:17.657158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23330d31-5d1a-4f0c-a27f-3ebaa3d91035 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:17.657158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23330d31-5d1a-4f0c-a27f-3ebaa3d91035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:17.697293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-23330d31-5d1a-4f0c-a27f-3ebaa3d91035 None None] MFA Rules not processed for user `831f85e5678248b6b2991b8ac3a5e2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:17.698888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-23330d31-5d1a-4f0c-a27f-3ebaa3d91035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:17.841661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1386/2775] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:17.862039 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afef36b-969d-4214-b379-0bef87b481a7 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:17.866402 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7afef36b-969d-4214-b379-0bef87b481a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:17.912175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878112b5-fd30-44c0-af3b-32a6b121c90e None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:17.912175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878112b5-fd30-44c0-af3b-32a6b121c90e None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:17.912942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878112b5-fd30-44c0-af3b-32a6b121c90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:17.952390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-878112b5-fd30-44c0-af3b-32a6b121c90e None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:17.953593 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-878112b5-fd30-44c0-af3b-32a6b121c90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:18.010041 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afef36b-969d-4214-b379-0bef87b481a7 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:18.014621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7c741-2366-4591-b74b-1a29ed655028 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:18.015291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c741-2366-4591-b74b-1a29ed655028 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:18.016507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c741-2366-4591-b74b-1a29ed655028 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:18.016507 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7c741-2366-4591-b74b-1a29ed655028 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:18.026933 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7c741-2366-4591-b74b-1a29ed655028 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:18.028437 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7c741-2366-4591-b74b-1a29ed655028 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:18.036118 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1387/2776] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:19:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:18.039957 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1390/2777] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:18.067198 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878112b5-fd30-44c0-af3b-32a6b121c90e None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:18.068789 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-878112b5-fd30-44c0-af3b-32a6b121c90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:18.092330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:18.244120 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878112b5-fd30-44c0-af3b-32a6b121c90e None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:18.257450 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4a1c3-93ab-4e45-bde5-5d26293c00d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:18.258269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4a1c3-93ab-4e45-bde5-5d26293c00d4 service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:18.258809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4a1c3-93ab-4e45-bde5-5d26293c00d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:18.259185 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4a1c3-93ab-4e45-bde5-5d26293c00d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:18.280454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4a1c3-93ab-4e45-bde5-5d26293c00d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:18.284474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4a1c3-93ab-4e45-bde5-5d26293c00d4 service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:18.300717 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1388/2778] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:19:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:19:21.005684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5e213-05ff-46b0-a145-6860ae59462c None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:21.006607 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5e213-05ff-46b0-a145-6860ae59462c None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:21.007254 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5e213-05ff-46b0-a145-6860ae59462c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:21.074407 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-86a5e213-05ff-46b0-a145-6860ae59462c None None] MFA Rules not processed for user `831f85e5678248b6b2991b8ac3a5e2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:21.075716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-86a5e213-05ff-46b0-a145-6860ae59462c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:21.084369 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1391/2779] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:28.295271 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=26 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:30.400619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4a1c3-93ab-4e45-bde5-5d26293c00d4 service swift] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:30.411852 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4a1c3-93ab-4e45-bde5-5d26293c00d4 service swift] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:30.418738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a66129-c3da-4d9b-aec9-848603aa3a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:30.420474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a66129-c3da-4d9b-aec9-848603aa3a3d service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:30.420908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a66129-c3da-4d9b-aec9-848603aa3a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:30.421309 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a66129-c3da-4d9b-aec9-848603aa3a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:30.424363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-85a66129-c3da-4d9b-aec9-848603aa3a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:30.438800 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a66129-c3da-4d9b-aec9-848603aa3a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:30.440896 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a66129-c3da-4d9b-aec9-848603aa3a3d service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:30.449694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1389/2780] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:31.056794 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=31 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:31.242376 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5e213-05ff-46b0-a145-6860ae59462c None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:31.244050 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-86a5e213-05ff-46b0-a145-6860ae59462c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:31.334558 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5e213-05ff-46b0-a145-6860ae59462c None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:31.340388 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4695c-cdfb-4235-8836-4ececb921c39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:31.341067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4695c-cdfb-4235-8836-4ececb921c39 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:31.341366 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4695c-cdfb-4235-8836-4ececb921c39 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:31.341653 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4695c-cdfb-4235-8836-4ececb921c39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:31.343638 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-72d4695c-cdfb-4235-8836-4ececb921c39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:31.357808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4695c-cdfb-4235-8836-4ececb921c39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:31.359482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4695c-cdfb-4235-8836-4ececb921c39 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:31.372538 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1392/2781] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:39.805196 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a66129-c3da-4d9b-aec9-848603aa3a3d service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:39.807096 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-85a66129-c3da-4d9b-aec9-848603aa3a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:39.822676 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a66129-c3da-4d9b-aec9-848603aa3a3d service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:39.828059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:39.828950 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:39.829234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:39.829511 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] PATH_INFO: `/v3/users/38c6d95788a345ef8a05eff6098bb2c2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:39.830999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38c6d95788a345ef8a05eff6098bb2c2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:39.833062 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:39.885279 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38c6d95788a345ef8a05eff6098bb2c2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:39.885958 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38c6d95788a345ef8a05eff6098bb2c2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:39.894110 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38c6d95788a345ef8a05eff6098bb2c2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:39.902948 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38c6d95788a345ef8a05eff6098bb2c2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:39.903577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38c6d95788a345ef8a05eff6098bb2c2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:39.930764 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1390/2782] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:19:39 2022] DELETE /identity/v3/users/38c6d95788a345ef8a05eff6098bb2c2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:39.935399 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f59876-1141-478f-a8c3-522034ab3fde None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:39.935903 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f59876-1141-478f-a8c3-522034ab3fde None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:39.936031 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f59876-1141-478f-a8c3-522034ab3fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:39.956327 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-32f59876-1141-478f-a8c3-522034ab3fde None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:39.957544 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-32f59876-1141-478f-a8c3-522034ab3fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:40.021576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1393/2783] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:40.033791 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:40.035668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:40.102182 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a40ebd-1e85-4c8a-8ee0-728f2b82b026 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:40.105099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2794f7a-6d78-463e-8dee-7e87b6c7a338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:40.105772 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2794f7a-6d78-463e-8dee-7e87b6c7a338 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:40.106061 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2794f7a-6d78-463e-8dee-7e87b6c7a338 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:40.106345 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2794f7a-6d78-463e-8dee-7e87b6c7a338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:40.117676 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2794f7a-6d78-463e-8dee-7e87b6c7a338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:40.119147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2794f7a-6d78-463e-8dee-7e87b6c7a338 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:40.131338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1391/2784] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:40.434619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f59876-1141-478f-a8c3-522034ab3fde None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:40.437177 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-32f59876-1141-478f-a8c3-522034ab3fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:40.459436 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f59876-1141-478f-a8c3-522034ab3fde None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:40.472896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1835b9-d6b8-483e-87b6-53264cc7b26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:40.473789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1835b9-d6b8-483e-87b6-53264cc7b26b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:40.474077 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1835b9-d6b8-483e-87b6-53264cc7b26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:40.474352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1835b9-d6b8-483e-87b6-53264cc7b26b admin admin] PATH_INFO: `/v3/projects/92623b5296c4468c97f4c975a54d8495` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:40.475846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1835b9-d6b8-483e-87b6-53264cc7b26b admin admin] RBAC: Authorizing `identity:delete_project(project_id=92623b5296c4468c97f4c975a54d8495)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:40.477833 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1835b9-d6b8-483e-87b6-53264cc7b26b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:40.561043 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-de1835b9-d6b8-483e-87b6-53264cc7b26b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92623b5296c4468c97f4c975a54d8495', 'request_id': 'req-de1835b9-d6b8-483e-87b6-53264cc7b26b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:40.569513 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-de1835b9-d6b8-483e-87b6-53264cc7b26b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92623b5296c4468c97f4c975a54d8495', 'request_id': 'req-de1835b9-d6b8-483e-87b6-53264cc7b26b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:40.579148 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-de1835b9-d6b8-483e-87b6-53264cc7b26b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92623b5296c4468c97f4c975a54d8495', 'request_id': 'req-de1835b9-d6b8-483e-87b6-53264cc7b26b'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:40.603763 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-de1835b9-d6b8-483e-87b6-53264cc7b26b admin admin] The token cache is being invalidate because project 92623b5296c4468c97f4c975a54d8495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:19:40.604099 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-de1835b9-d6b8-483e-87b6-53264cc7b26b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:40.604849 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-de1835b9-d6b8-483e-87b6-53264cc7b26b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:19:40.612156 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc2871-2591-4a63-bf59-82e8b0fd6fd6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:40.612578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc2871-2591-4a63-bf59-82e8b0fd6fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:40.612738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc2871-2591-4a63-bf59-82e8b0fd6fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:40.617280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1394/2785] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:19:40 2022] DELETE /identity/v3/projects/92623b5296c4468c97f4c975a54d8495 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:40.646933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-a9dc2871-2591-4a63-bf59-82e8b0fd6fd6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:40.650779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-a9dc2871-2591-4a63-bf59-82e8b0fd6fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:40.761579 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1392/2786] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:40.770410 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc0952-0f90-4928-ad9a-70f5505d99f6 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:40.770702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc0952-0f90-4928-ad9a-70f5505d99f6 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:40.770989 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc0952-0f90-4928-ad9a-70f5505d99f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:40.791711 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-69bc0952-0f90-4928-ad9a-70f5505d99f6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:40.792964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-69bc0952-0f90-4928-ad9a-70f5505d99f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:40.803027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1395/2787] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:40.807480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc2871-2591-4a63-bf59-82e8b0fd6fd6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:40.812874 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:40.823625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc0952-0f90-4928-ad9a-70f5505d99f6 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:40.824710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc2871-2591-4a63-bf59-82e8b0fd6fd6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:40.825096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-69bc0952-0f90-4928-ad9a-70f5505d99f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:40.829553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a407f2-3b1f-430c-97b6-8ac1a2b85119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:40.830464 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a407f2-3b1f-430c-97b6-8ac1a2b85119 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:40.830755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a407f2-3b1f-430c-97b6-8ac1a2b85119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:40.831050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a407f2-3b1f-430c-97b6-8ac1a2b85119 admin admin] PATH_INFO: `/v3/domains` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:40.831562 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:40.831898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a407f2-3b1f-430c-97b6-8ac1a2b85119 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80183) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:19:40.832439 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a407f2-3b1f-430c-97b6-8ac1a2b85119 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:40.834163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a407f2-3b1f-430c-97b6-8ac1a2b85119 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:40.850600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1393/2788] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:19:40.856297 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a407f2-3b1f-430c-97b6-8ac1a2b85119 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:40.871330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a407f2-3b1f-430c-97b6-8ac1a2b85119 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:40.877441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdfc6a-8f54-49df-b291-fa8db96d615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:40.878221 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdfc6a-8f54-49df-b291-fa8db96d615f admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:40.878570 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdfc6a-8f54-49df-b291-fa8db96d615f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:40.878920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdfc6a-8f54-49df-b291-fa8db96d615f admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:40.879879 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdfc6a-8f54-49df-b291-fa8db96d615f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:40.882046 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdfc6a-8f54-49df-b291-fa8db96d615f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:40.913495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1394/2789] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:19:40 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:19:40.917559 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdfc6a-8f54-49df-b291-fa8db96d615f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:40.928455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc0952-0f90-4928-ad9a-70f5505d99f6 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:41.002243 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f81933-6b41-4cdc-9df5-76834b44b88b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:41.002920 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdfc6a-8f54-49df-b291-fa8db96d615f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:41.003048 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f81933-6b41-4cdc-9df5-76834b44b88b service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:41.004584 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f81933-6b41-4cdc-9df5-76834b44b88b service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:41.005044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f81933-6b41-4cdc-9df5-76834b44b88b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:41.007913 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-67f81933-6b41-4cdc-9df5-76834b44b88b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:41.008985 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371fede-75cd-4211-a5ac-75844875c953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:41.009695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371fede-75cd-4211-a5ac-75844875c953 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:41.009983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371fede-75cd-4211-a5ac-75844875c953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:41.010263 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371fede-75cd-4211-a5ac-75844875c953 admin admin] PATH_INFO: `/v3/users` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:41.010955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371fede-75cd-4211-a5ac-75844875c953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:41.012938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371fede-75cd-4211-a5ac-75844875c953 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:41.022680 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f81933-6b41-4cdc-9df5-76834b44b88b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:41.024726 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f81933-6b41-4cdc-9df5-76834b44b88b service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:41.040249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1395/2790] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:19:40 2022] POST /identity/v3/users => generated 282 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:19:41.040535 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1396/2791] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:19:41.042022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371fede-75cd-4211-a5ac-75844875c953 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:41.056897 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371fede-75cd-4211-a5ac-75844875c953 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:41.061927 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8817ec-b5b9-49df-ad6e-7aafbd6cabf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:41.062616 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8817ec-b5b9-49df-ad6e-7aafbd6cabf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:41.062893 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8817ec-b5b9-49df-ad6e-7aafbd6cabf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:41.063178 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8817ec-b5b9-49df-ad6e-7aafbd6cabf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:41.063965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8817ec-b5b9-49df-ad6e-7aafbd6cabf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:41.066164 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8817ec-b5b9-49df-ad6e-7aafbd6cabf3 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:41.078139 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1396/2792] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:19:41.082450 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f81933-6b41-4cdc-9df5-76834b44b88b service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:41.094255 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f81933-6b41-4cdc-9df5-76834b44b88b service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:41.099063 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c42ca-0dbe-4b52-bfdd-89045e8db0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:41.099826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c42ca-0dbe-4b52-bfdd-89045e8db0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:41.100115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c42ca-0dbe-4b52-bfdd-89045e8db0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:41.100400 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c42ca-0dbe-4b52-bfdd-89045e8db0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:41.101159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c42ca-0dbe-4b52-bfdd-89045e8db0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:41.102883 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c42ca-0dbe-4b52-bfdd-89045e8db0c1 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:41.114069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1397/2793] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:19:41.118652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8817ec-b5b9-49df-ad6e-7aafbd6cabf3 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:41.129574 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8817ec-b5b9-49df-ad6e-7aafbd6cabf3 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:41.134496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a31d3a-95aa-4e3e-8547-9570a703b5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:41.135407 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a31d3a-95aa-4e3e-8547-9570a703b5e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:41.135713 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a31d3a-95aa-4e3e-8547-9570a703b5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:41.135999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a31d3a-95aa-4e3e-8547-9570a703b5e8 admin admin] PATH_INFO: `/v3/projects/653dfa0ed71b439694cbd2116cad942c/users/1b230c994ebc436fb42354880c5e1d7c/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:41.152217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a31d3a-95aa-4e3e-8547-9570a703b5e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=653dfa0ed71b439694cbd2116cad942c, user_id=1b230c994ebc436fb42354880c5e1d7c, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:41.154837 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a31d3a-95aa-4e3e-8547-9570a703b5e8 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:41.170408 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1397/2794] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:19:41 2022] PUT /identity/v3/projects/653dfa0ed71b439694cbd2116cad942c/users/1b230c994ebc436fb42354880c5e1d7c/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:19:41.183377 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f6074-e01e-4238-a7bc-8dd9b371e11b None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:41.183691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f6074-e01e-4238-a7bc-8dd9b371e11b None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:41.183974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f6074-e01e-4238-a7bc-8dd9b371e11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:41.219909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ed4f6074-e01e-4238-a7bc-8dd9b371e11b None None] MFA Rules not processed for user `1b230c994ebc436fb42354880c5e1d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:41.221879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ed4f6074-e01e-4238-a7bc-8dd9b371e11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:41.319064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1398/2795] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:41 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:41.338240 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a31d3a-95aa-4e3e-8547-9570a703b5e8 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:41.341496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f9a31d3a-95aa-4e3e-8547-9570a703b5e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:41.420914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a31d3a-95aa-4e3e-8547-9570a703b5e8 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:41.424913 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca8a40-6dae-42f5-ac6e-199121ae725b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:41.425675 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca8a40-6dae-42f5-ac6e-199121ae725b service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:41.425911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca8a40-6dae-42f5-ac6e-199121ae725b service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:41.426227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca8a40-6dae-42f5-ac6e-199121ae725b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:41.436480 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca8a40-6dae-42f5-ac6e-199121ae725b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:41.438047 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca8a40-6dae-42f5-ac6e-199121ae725b service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:41.448869 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1398/2796] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:41 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:51.313966 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=98 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:51.444777 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=84,keystone:INSERT=5 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:19:53.218197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8a59c-f400-47b6-be1a-47fbd4e8d557 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:53.218973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8a59c-f400-47b6-be1a-47fbd4e8d557 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:53.218973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8a59c-f400-47b6-be1a-47fbd4e8d557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:53.238647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-bff8a59c-f400-47b6-be1a-47fbd4e8d557 None None] MFA Rules not processed for user `831f85e5678248b6b2991b8ac3a5e2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:19:53.240496 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bff8a59c-f400-47b6-be1a-47fbd4e8d557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:53.311614 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1399/2797] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:19:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:19:53.589189 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca8a40-6dae-42f5-ac6e-199121ae725b service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:53.591168 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-7bca8a40-6dae-42f5-ac6e-199121ae725b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:53.665825 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca8a40-6dae-42f5-ac6e-199121ae725b service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:53.670889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f58a48-e6ce-4ef8-91c5-b5a1981c7163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:53.672017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f58a48-e6ce-4ef8-91c5-b5a1981c7163 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:53.672502 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f58a48-e6ce-4ef8-91c5-b5a1981c7163 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:53.672991 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f58a48-e6ce-4ef8-91c5-b5a1981c7163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:53.676328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b8f58a48-e6ce-4ef8-91c5-b5a1981c7163 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:53.687842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f58a48-e6ce-4ef8-91c5-b5a1981c7163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:53.690107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f58a48-e6ce-4ef8-91c5-b5a1981c7163 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:53.702578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1399/2798] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:55.188426 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8a59c-f400-47b6-be1a-47fbd4e8d557 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:55.191127 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-bff8a59c-f400-47b6-be1a-47fbd4e8d557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:55.208118 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8a59c-f400-47b6-be1a-47fbd4e8d557 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:55.212405 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaccbed-fcce-46c6-90df-90d0be669835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:55.213178 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaccbed-fcce-46c6-90df-90d0be669835 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:55.213509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaccbed-fcce-46c6-90df-90d0be669835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:55.213835 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaccbed-fcce-46c6-90df-90d0be669835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:55.216484 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-7aaccbed-fcce-46c6-90df-90d0be669835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:55.315218 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaccbed-fcce-46c6-90df-90d0be669835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:55.317067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaccbed-fcce-46c6-90df-90d0be669835 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:55.333870 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1400/2799] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:19:57.938380 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f58a48-e6ce-4ef8-91c5-b5a1981c7163 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:19:58.055336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f58a48-e6ce-4ef8-91c5-b5a1981c7163 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:19:58.090828 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e028d-de95-44fa-a7f4-2a07de7d579e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:19:58.092022 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e028d-de95-44fa-a7f4-2a07de7d579e service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:19:58.092331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e028d-de95-44fa-a7f4-2a07de7d579e service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:19:58.092804 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e028d-de95-44fa-a7f4-2a07de7d579e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:19:58.100165 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-721e028d-de95-44fa-a7f4-2a07de7d579e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:19:58.132156 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e028d-de95-44fa-a7f4-2a07de7d579e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:19:58.134872 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e028d-de95-44fa-a7f4-2a07de7d579e service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:19:58.166821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1400/2800] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:19:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:05.325494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=55 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:08.154220 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=60 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:22.245681 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd31c9-e579-41b6-b9e1-fe61e6f687b0 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:22.246158 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd31c9-e579-41b6-b9e1-fe61e6f687b0 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:22.246261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd31c9-e579-41b6-b9e1-fe61e6f687b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:22.265309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-0acd31c9-e579-41b6-b9e1-fe61e6f687b0 None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:22.266527 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-0acd31c9-e579-41b6-b9e1-fe61e6f687b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:22.322770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1401/2801] 10.222.0.75 () {58 vars in 1009 bytes} [Thu Jul 14 11:20:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:22.369580 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e028d-de95-44fa-a7f4-2a07de7d579e service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:22.371015 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-721e028d-de95-44fa-a7f4-2a07de7d579e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:22.444431 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e028d-de95-44fa-a7f4-2a07de7d579e service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:22.452316 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590aca4c-2676-4328-b4ef-a0654e00e01f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:22.453433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590aca4c-2676-4328-b4ef-a0654e00e01f service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:22.453947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590aca4c-2676-4328-b4ef-a0654e00e01f service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:22.454455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590aca4c-2676-4328-b4ef-a0654e00e01f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:22.465725 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590aca4c-2676-4328-b4ef-a0654e00e01f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:22.468363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590aca4c-2676-4328-b4ef-a0654e00e01f service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:22.477824 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1401/2802] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:20:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:20:30.640429 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd31c9-e579-41b6-b9e1-fe61e6f687b0 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:30.722169 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd31c9-e579-41b6-b9e1-fe61e6f687b0 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:30.726939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:30.727899 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:30.728187 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:30.728468 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] PATH_INFO: `/v3/users/1b230c994ebc436fb42354880c5e1d7c` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:30.729779 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b230c994ebc436fb42354880c5e1d7c)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:30.731851 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:30.792222 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b230c994ebc436fb42354880c5e1d7c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:30.803978 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b230c994ebc436fb42354880c5e1d7c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:30.804854 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b230c994ebc436fb42354880c5e1d7c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:30.817769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b230c994ebc436fb42354880c5e1d7c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:30.818856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b230c994ebc436fb42354880c5e1d7c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:30.846165 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1402/2803] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:20:30 2022] DELETE /identity/v3/users/1b230c994ebc436fb42354880c5e1d7c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:30.849059 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ba033-ee0e-465d-9ebb-d15903d16887 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:30.849657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ba033-ee0e-465d-9ebb-d15903d16887 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:30.850050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ba033-ee0e-465d-9ebb-d15903d16887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:30.882761 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-8d3ba033-ee0e-465d-9ebb-d15903d16887 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:30.884817 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8d3ba033-ee0e-465d-9ebb-d15903d16887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:30.990193 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1402/2804] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:31.004568 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:31.007856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:31.057901 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:31.169654 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2f29b-491f-4a7b-b883-a2e3efae0472 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:31.176489 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90101b62-d2b7-410f-9e63-6e463a449ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:31.177673 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90101b62-d2b7-410f-9e63-6e463a449ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:31.178185 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90101b62-d2b7-410f-9e63-6e463a449ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:31.179429 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90101b62-d2b7-410f-9e63-6e463a449ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:31.205379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90101b62-d2b7-410f-9e63-6e463a449ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:31.205379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90101b62-d2b7-410f-9e63-6e463a449ea4 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:31.221952 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1403/2805] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:31.967766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ba033-ee0e-465d-9ebb-d15903d16887 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:31.969418 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8d3ba033-ee0e-465d-9ebb-d15903d16887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:32.004217 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ba033-ee0e-465d-9ebb-d15903d16887 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:32.014368 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f17865-6c6f-49d0-9bee-c94e4d048707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:32.018213 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17865-6c6f-49d0-9bee-c94e4d048707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:32.019326 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17865-6c6f-49d0-9bee-c94e4d048707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:32.020030 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17865-6c6f-49d0-9bee-c94e4d048707 admin admin] PATH_INFO: `/v3/projects/653dfa0ed71b439694cbd2116cad942c` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:32.024698 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f17865-6c6f-49d0-9bee-c94e4d048707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=653dfa0ed71b439694cbd2116cad942c)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:32.029774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f17865-6c6f-49d0-9bee-c94e4d048707 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:32.138565 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-58f17865-6c6f-49d0-9bee-c94e4d048707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '653dfa0ed71b439694cbd2116cad942c', 'request_id': 'req-58f17865-6c6f-49d0-9bee-c94e4d048707'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:32.152633 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-58f17865-6c6f-49d0-9bee-c94e4d048707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '653dfa0ed71b439694cbd2116cad942c', 'request_id': 'req-58f17865-6c6f-49d0-9bee-c94e4d048707'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:32.164940 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-58f17865-6c6f-49d0-9bee-c94e4d048707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '653dfa0ed71b439694cbd2116cad942c', 'request_id': 'req-58f17865-6c6f-49d0-9bee-c94e4d048707'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:32.193029 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-58f17865-6c6f-49d0-9bee-c94e4d048707 admin admin] The token cache is being invalidate because project 653dfa0ed71b439694cbd2116cad942c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:20:32.193588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-58f17865-6c6f-49d0-9bee-c94e4d048707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:32.194591 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-58f17865-6c6f-49d0-9bee-c94e4d048707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:32.197447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1403/2806] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:20:31 2022] DELETE /identity/v3/projects/653dfa0ed71b439694cbd2116cad942c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:32.204024 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6ebca-dace-4a33-ba66-b8f130147d63 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:32.204826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6ebca-dace-4a33-ba66-b8f130147d63 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:32.204826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6ebca-dace-4a33-ba66-b8f130147d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:32.227517 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-49f6ebca-dace-4a33-ba66-b8f130147d63 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:32.228744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-49f6ebca-dace-4a33-ba66-b8f130147d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:32.312047 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1404/2807] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:32.320915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f81c8c-6a8a-4708-b4e6-2996a3bf1257 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:32.321411 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f81c8c-6a8a-4708-b4e6-2996a3bf1257 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:32.321779 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f81c8c-6a8a-4708-b4e6-2996a3bf1257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:32.348495 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-07f81c8c-6a8a-4708-b4e6-2996a3bf1257 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:32.350260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-07f81c8c-6a8a-4708-b4e6-2996a3bf1257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:32.358416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1404/2808] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:32.362747 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6ebca-dace-4a33-ba66-b8f130147d63 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:32.373796 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6ebca-dace-4a33-ba66-b8f130147d63 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:32.379062 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6f766-d799-415f-bdd7-09c1472fd787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:32.379781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6f766-d799-415f-bdd7-09c1472fd787 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:32.380066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6f766-d799-415f-bdd7-09c1472fd787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:32.380340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6f766-d799-415f-bdd7-09c1472fd787 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:32.381070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6f766-d799-415f-bdd7-09c1472fd787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:20:32.381365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6f766-d799-415f-bdd7-09c1472fd787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:32.382989 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6f766-d799-415f-bdd7-09c1472fd787 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:32.396406 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1405/2809] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:20:32.400657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f81c8c-6a8a-4708-b4e6-2996a3bf1257 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:32.410940 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f81c8c-6a8a-4708-b4e6-2996a3bf1257 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:32.415732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ca54b-57ab-441e-b41f-8737ba6fec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:32.416412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ca54b-57ab-441e-b41f-8737ba6fec47 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:32.416700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ca54b-57ab-441e-b41f-8737ba6fec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:32.416979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ca54b-57ab-441e-b41f-8737ba6fec47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:32.417688 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ca54b-57ab-441e-b41f-8737ba6fec47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:32.419661 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ca54b-57ab-441e-b41f-8737ba6fec47 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:32.423545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:32.451935 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1405/2810] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:20:32 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:20:32.456509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6f766-d799-415f-bdd7-09c1472fd787 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:32.468526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6f766-d799-415f-bdd7-09c1472fd787 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:32.551428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99771188-6a83-4123-a86d-cf2e07e57f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:32.551428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99771188-6a83-4123-a86d-cf2e07e57f0f admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:32.551428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99771188-6a83-4123-a86d-cf2e07e57f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:32.551428 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99771188-6a83-4123-a86d-cf2e07e57f0f admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:32.552016 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99771188-6a83-4123-a86d-cf2e07e57f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:32.553574 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99771188-6a83-4123-a86d-cf2e07e57f0f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:32.584705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1406/2811] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:20:32 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:20:32.589444 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ca54b-57ab-441e-b41f-8737ba6fec47 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:32.608300 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ca54b-57ab-441e-b41f-8737ba6fec47 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:32.614323 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895535b-72e7-4be9-b9f3-e923a43ae19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:32.615187 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895535b-72e7-4be9-b9f3-e923a43ae19b admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:32.615841 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895535b-72e7-4be9-b9f3-e923a43ae19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:32.616539 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895535b-72e7-4be9-b9f3-e923a43ae19b admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:32.617447 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895535b-72e7-4be9-b9f3-e923a43ae19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:32.620298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895535b-72e7-4be9-b9f3-e923a43ae19b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:32.640771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1406/2812] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:20:32.647283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99771188-6a83-4123-a86d-cf2e07e57f0f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:32.665863 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99771188-6a83-4123-a86d-cf2e07e57f0f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:32.677900 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fe1bf-26df-47a4-9d5f-441549706d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:32.679056 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fe1bf-26df-47a4-9d5f-441549706d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:32.680155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fe1bf-26df-47a4-9d5f-441549706d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:32.680629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fe1bf-26df-47a4-9d5f-441549706d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:32.682282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fe1bf-26df-47a4-9d5f-441549706d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:32.685302 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fe1bf-26df-47a4-9d5f-441549706d3b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:32.707442 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1407/2813] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:20:32.711816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895535b-72e7-4be9-b9f3-e923a43ae19b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:32.725979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895535b-72e7-4be9-b9f3-e923a43ae19b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:32.731729 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a2ab4-e13f-4d02-9b73-5c77f436ab3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:32.735441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a2ab4-e13f-4d02-9b73-5c77f436ab3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:32.735441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a2ab4-e13f-4d02-9b73-5c77f436ab3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:32.735441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a2ab4-e13f-4d02-9b73-5c77f436ab3c admin admin] PATH_INFO: `/v3/projects/183612235b834368be67c02434f45d36/users/6d9987a8042840caa38d2112fcce404a/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:32.768009 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a2ab4-e13f-4d02-9b73-5c77f436ab3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=183612235b834368be67c02434f45d36, user_id=6d9987a8042840caa38d2112fcce404a, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:32.772231 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a2ab4-e13f-4d02-9b73-5c77f436ab3c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:32.787578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1407/2814] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:20:32 2022] PUT /identity/v3/projects/183612235b834368be67c02434f45d36/users/6d9987a8042840caa38d2112fcce404a/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:20:32.797009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce201f-af5e-4ee1-b594-a31dbd05255a None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:32.797009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce201f-af5e-4ee1-b594-a31dbd05255a None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:32.797009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce201f-af5e-4ee1-b594-a31dbd05255a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:32.828821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-45ce201f-af5e-4ee1-b594-a31dbd05255a None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:32.830096 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-45ce201f-af5e-4ee1-b594-a31dbd05255a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:32.869059 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:32.918280 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1408/2815] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:20:32.935727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a2ab4-e13f-4d02-9b73-5c77f436ab3c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:32.938040 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f81a2ab4-e13f-4d02-9b73-5c77f436ab3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:33.011942 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a2ab4-e13f-4d02-9b73-5c77f436ab3c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:33.016285 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec66fb7-f2ec-4fe8-be81-af5439fe7903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:33.017035 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec66fb7-f2ec-4fe8-be81-af5439fe7903 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:33.017399 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec66fb7-f2ec-4fe8-be81-af5439fe7903 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:33.017922 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec66fb7-f2ec-4fe8-be81-af5439fe7903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:33.030349 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec66fb7-f2ec-4fe8-be81-af5439fe7903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:33.031868 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec66fb7-f2ec-4fe8-be81-af5439fe7903 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:33.045477 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1408/2816] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:34.002890 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01af51-7e22-4059-85e3-cae24cad25fb None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:34.003524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01af51-7e22-4059-85e3-cae24cad25fb None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:34.003662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01af51-7e22-4059-85e3-cae24cad25fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:34.063730 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ee01af51-7e22-4059-85e3-cae24cad25fb None None] MFA Rules not processed for user `6d9987a8042840caa38d2112fcce404a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:34.064839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ee01af51-7e22-4059-85e3-cae24cad25fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:34.171716 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1409/2817] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:20:34.184476 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec66fb7-f2ec-4fe8-be81-af5439fe7903 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:34.187266 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-aec66fb7-f2ec-4fe8-be81-af5439fe7903 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:34.269856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec66fb7-f2ec-4fe8-be81-af5439fe7903 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:34.274378 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452335dc-6262-4c26-b7c3-f6b2bd085559 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:34.277292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452335dc-6262-4c26-b7c3-f6b2bd085559 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:34.277292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452335dc-6262-4c26-b7c3-f6b2bd085559 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:34.277292 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452335dc-6262-4c26-b7c3-f6b2bd085559 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:34.288420 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452335dc-6262-4c26-b7c3-f6b2bd085559 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:34.290720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452335dc-6262-4c26-b7c3-f6b2bd085559 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:34.307605 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1409/2818] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:35.836708 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dfea5-a1c3-450b-81bf-cc93e14b5319 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:35.837700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dfea5-a1c3-450b-81bf-cc93e14b5319 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:35.837700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dfea5-a1c3-450b-81bf-cc93e14b5319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:35.864119 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-be4dfea5-a1c3-450b-81bf-cc93e14b5319 None None] MFA Rules not processed for user `6d9987a8042840caa38d2112fcce404a`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:35.866466 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-be4dfea5-a1c3-450b-81bf-cc93e14b5319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:35.877420 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1410/2819] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:20:37.519608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452335dc-6262-4c26-b7c3-f6b2bd085559 service cinder] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:37.522180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-452335dc-6262-4c26-b7c3-f6b2bd085559 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:37.619993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:37.638782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452335dc-6262-4c26-b7c3-f6b2bd085559 service cinder] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:37.647734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12d883-5d55-4f87-b2de-478aa33a9d82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:37.648866 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12d883-5d55-4f87-b2de-478aa33a9d82 service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:37.649993 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12d883-5d55-4f87-b2de-478aa33a9d82 service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:37.650490 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12d883-5d55-4f87-b2de-478aa33a9d82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:37.654274 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cf12d883-5d55-4f87-b2de-478aa33a9d82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:37.685694 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12d883-5d55-4f87-b2de-478aa33a9d82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:37.688700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12d883-5d55-4f87-b2de-478aa33a9d82 service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:37.706677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1410/2820] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:39.807772 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdbdcb-c1a9-4972-8c9e-69c424a33371 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:39.808481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdbdcb-c1a9-4972-8c9e-69c424a33371 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:39.808481 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdbdcb-c1a9-4972-8c9e-69c424a33371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:39.827345 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-4bfdbdcb-c1a9-4972-8c9e-69c424a33371 None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:39.828834 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4bfdbdcb-c1a9-4972-8c9e-69c424a33371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:39.890267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1411/2821] 10.222.0.75 () {58 vars in 967 bytes} [Thu Jul 14 11:20:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:39.905234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12d883-5d55-4f87-b2de-478aa33a9d82 service placement] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:39.907156 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-cf12d883-5d55-4f87-b2de-478aa33a9d82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:40.008554 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12d883-5d55-4f87-b2de-478aa33a9d82 service placement] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:40.015698 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d9dc0-5365-4e2f-9b8a-f413891e2cd0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:40.016809 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d9dc0-5365-4e2f-9b8a-f413891e2cd0 service swift] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:40.017326 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d9dc0-5365-4e2f-9b8a-f413891e2cd0 service swift] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:40.018108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d9dc0-5365-4e2f-9b8a-f413891e2cd0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:40.027805 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d9dc0-5365-4e2f-9b8a-f413891e2cd0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:40.029446 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d9dc0-5365-4e2f-9b8a-f413891e2cd0 service swift] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:40.041144 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1411/2822] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:20:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:20:42.136916 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdbdcb-c1a9-4972-8c9e-69c424a33371 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:42.138855 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-4bfdbdcb-c1a9-4972-8c9e-69c424a33371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:42.154285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdbdcb-c1a9-4972-8c9e-69c424a33371 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:42.158875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7732fe-4a1c-4f87-a46b-e83aca803260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:42.159811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7732fe-4a1c-4f87-a46b-e83aca803260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:42.160097 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7732fe-4a1c-4f87-a46b-e83aca803260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:42.160381 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7732fe-4a1c-4f87-a46b-e83aca803260 admin admin] PATH_INFO: `/v3/users/831f85e5678248b6b2991b8ac3a5e2b6` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:42.161591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7732fe-4a1c-4f87-a46b-e83aca803260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=831f85e5678248b6b2991b8ac3a5e2b6)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:42.163580 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7732fe-4a1c-4f87-a46b-e83aca803260 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:42.215615 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-de7732fe-4a1c-4f87-a46b-e83aca803260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '831f85e5678248b6b2991b8ac3a5e2b6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:42.223744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-de7732fe-4a1c-4f87-a46b-e83aca803260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '831f85e5678248b6b2991b8ac3a5e2b6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:42.224375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-de7732fe-4a1c-4f87-a46b-e83aca803260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '831f85e5678248b6b2991b8ac3a5e2b6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:42.226637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:42.233304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-de7732fe-4a1c-4f87-a46b-e83aca803260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '831f85e5678248b6b2991b8ac3a5e2b6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:42.234389 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-de7732fe-4a1c-4f87-a46b-e83aca803260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '831f85e5678248b6b2991b8ac3a5e2b6'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:42.253922 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1412/2823] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:20:42 2022] DELETE /identity/v3/users/831f85e5678248b6b2991b8ac3a5e2b6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:42.426699 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d9dc0-5365-4e2f-9b8a-f413891e2cd0 service swift] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:42.442952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d9dc0-5365-4e2f-9b8a-f413891e2cd0 service swift] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:42.503306 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0caa3d-3c18-4279-b2f5-0254039000a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:42.504248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0caa3d-3c18-4279-b2f5-0254039000a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:42.504554 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0caa3d-3c18-4279-b2f5-0254039000a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:42.504831 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0caa3d-3c18-4279-b2f5-0254039000a6 admin admin] PATH_INFO: `/v3/projects/442d027d621e40e2800fd6467f2bb60d` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:42.506192 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0caa3d-3c18-4279-b2f5-0254039000a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=442d027d621e40e2800fd6467f2bb60d)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:42.508181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0caa3d-3c18-4279-b2f5-0254039000a6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:42.556334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2f0caa3d-3c18-4279-b2f5-0254039000a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '442d027d621e40e2800fd6467f2bb60d', 'request_id': 'req-2f0caa3d-3c18-4279-b2f5-0254039000a6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:42.562704 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2f0caa3d-3c18-4279-b2f5-0254039000a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '442d027d621e40e2800fd6467f2bb60d', 'request_id': 'req-2f0caa3d-3c18-4279-b2f5-0254039000a6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:42.569458 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2f0caa3d-3c18-4279-b2f5-0254039000a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '442d027d621e40e2800fd6467f2bb60d', 'request_id': 'req-2f0caa3d-3c18-4279-b2f5-0254039000a6'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:42.584815 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2f0caa3d-3c18-4279-b2f5-0254039000a6 admin admin] The token cache is being invalidate because project 442d027d621e40e2800fd6467f2bb60d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:20:42.585102 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2f0caa3d-3c18-4279-b2f5-0254039000a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:42.585770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-2f0caa3d-3c18-4279-b2f5-0254039000a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:42.587596 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1412/2824] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:20:42 2022] DELETE /identity/v3/projects/442d027d621e40e2800fd6467f2bb60d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:42.592774 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1aa8-bd15-49ae-94f1-2b48a5fac348 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:42.593069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1aa8-bd15-49ae-94f1-2b48a5fac348 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:42.593339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b1aa8-bd15-49ae-94f1-2b48a5fac348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:42.617171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-309b1aa8-bd15-49ae-94f1-2b48a5fac348 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:42.618398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-309b1aa8-bd15-49ae-94f1-2b48a5fac348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:42.683446 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1413/2825] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:20:42.691666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc14cd9-40bb-4721-9ab4-076a38bf2077 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:42.691955 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc14cd9-40bb-4721-9ab4-076a38bf2077 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:42.692234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc14cd9-40bb-4721-9ab4-076a38bf2077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:42.700586 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:42.713750 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-0cc14cd9-40bb-4721-9ab4-076a38bf2077 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:42.716498 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0cc14cd9-40bb-4721-9ab4-076a38bf2077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:42.725469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1413/2826] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:20:42.729625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b1aa8-bd15-49ae-94f1-2b48a5fac348 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:42.739969 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b1aa8-bd15-49ae-94f1-2b48a5fac348 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:42.744630 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe26ff-d0f0-4d62-9041-e3b31c7538fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:42.745304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe26ff-d0f0-4d62-9041-e3b31c7538fd admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:42.745589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe26ff-d0f0-4d62-9041-e3b31c7538fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:42.745871 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe26ff-d0f0-4d62-9041-e3b31c7538fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:42.746611 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe26ff-d0f0-4d62-9041-e3b31c7538fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:20:42.746919 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe26ff-d0f0-4d62-9041-e3b31c7538fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:42.748579 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe26ff-d0f0-4d62-9041-e3b31c7538fd admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:42.761824 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1414/2827] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:20:42.766298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc14cd9-40bb-4721-9ab4-076a38bf2077 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:42.776433 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc14cd9-40bb-4721-9ab4-076a38bf2077 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:42.781219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0150d5-3776-446b-8a12-bf69ab291f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:42.781887 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0150d5-3776-446b-8a12-bf69ab291f78 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:42.782175 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0150d5-3776-446b-8a12-bf69ab291f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:42.782461 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0150d5-3776-446b-8a12-bf69ab291f78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:42.783245 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0150d5-3776-446b-8a12-bf69ab291f78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:42.785303 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0150d5-3776-446b-8a12-bf69ab291f78 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:42.817867 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1414/2828] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:20:42 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:20:42.819279 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe26ff-d0f0-4d62-9041-e3b31c7538fd admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:42.830631 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe26ff-d0f0-4d62-9041-e3b31c7538fd admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:42.890509 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c93cb4-da11-440c-ad0c-5f64538d794f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:42.891192 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c93cb4-da11-440c-ad0c-5f64538d794f admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:42.891508 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c93cb4-da11-440c-ad0c-5f64538d794f admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:42.891789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c93cb4-da11-440c-ad0c-5f64538d794f admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:42.892566 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c93cb4-da11-440c-ad0c-5f64538d794f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:42.894519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c93cb4-da11-440c-ad0c-5f64538d794f admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:42.915811 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1415/2829] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:20:42 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:20:42.920289 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0150d5-3776-446b-8a12-bf69ab291f78 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:42.931161 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0150d5-3776-446b-8a12-bf69ab291f78 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:42.936032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f555c-b232-4cde-8df1-9684b1c737ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:42.936704 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f555c-b232-4cde-8df1-9684b1c737ee admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:42.936992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f555c-b232-4cde-8df1-9684b1c737ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:42.937291 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f555c-b232-4cde-8df1-9684b1c737ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:42.938000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f555c-b232-4cde-8df1-9684b1c737ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:42.939669 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f555c-b232-4cde-8df1-9684b1c737ee admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:42.951253 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1415/2830] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:20:42.956062 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c93cb4-da11-440c-ad0c-5f64538d794f admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:42.966636 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c93cb4-da11-440c-ad0c-5f64538d794f admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:42.972116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5efd78-fba6-44b5-a9e3-ec1ebe0d6164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:42.972776 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5efd78-fba6-44b5-a9e3-ec1ebe0d6164 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:42.973088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5efd78-fba6-44b5-a9e3-ec1ebe0d6164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:42.973360 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5efd78-fba6-44b5-a9e3-ec1ebe0d6164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:42.974098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5efd78-fba6-44b5-a9e3-ec1ebe0d6164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:42.975702 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5efd78-fba6-44b5-a9e3-ec1ebe0d6164 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:42.985939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1416/2831] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:20:42.989951 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f555c-b232-4cde-8df1-9684b1c737ee admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:43.000158 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f555c-b232-4cde-8df1-9684b1c737ee admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:43.004798 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac3d3b-4811-4763-844f-6be5536bfa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:43.005667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac3d3b-4811-4763-844f-6be5536bfa92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:43.005975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac3d3b-4811-4763-844f-6be5536bfa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:43.006270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac3d3b-4811-4763-844f-6be5536bfa92 admin admin] PATH_INFO: `/v3/projects/379620f3483c484789cde34d81d72220/users/95c687998a65476c85c467753883da55/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:43.022298 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac3d3b-4811-4763-844f-6be5536bfa92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=379620f3483c484789cde34d81d72220, user_id=95c687998a65476c85c467753883da55, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:43.025046 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac3d3b-4811-4763-844f-6be5536bfa92 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:43.041606 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1416/2832] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:20:42 2022] PUT /identity/v3/projects/379620f3483c484789cde34d81d72220/users/95c687998a65476c85c467753883da55/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:43.052661 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0c911-e1c1-4d14-baea-16a3588d5cc5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:43.053129 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0c911-e1c1-4d14-baea-16a3588d5cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:43.053699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0c911-e1c1-4d14-baea-16a3588d5cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:43.085698 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-33d0c911-e1c1-4d14-baea-16a3588d5cc5 None None] MFA Rules not processed for user `95c687998a65476c85c467753883da55`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:43.086920 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-33d0c911-e1c1-4d14-baea-16a3588d5cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:43.117960 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:43.159921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1417/2833] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:43.170708 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac3d3b-4811-4763-844f-6be5536bfa92 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:43.172739 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-20ac3d3b-4811-4763-844f-6be5536bfa92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:43.236974 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac3d3b-4811-4763-844f-6be5536bfa92 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:43.240855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904356e-3571-4dd0-b721-54cdbafb8c37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:43.241710 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904356e-3571-4dd0-b721-54cdbafb8c37 service cinder] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:43.241997 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904356e-3571-4dd0-b721-54cdbafb8c37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:43.242277 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904356e-3571-4dd0-b721-54cdbafb8c37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:43.251100 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904356e-3571-4dd0-b721-54cdbafb8c37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:43.252619 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904356e-3571-4dd0-b721-54cdbafb8c37 service cinder] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:43.262107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1417/2834] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:53.155973 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=14 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:53.257952 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:53.423172 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0c911-e1c1-4d14-baea-16a3588d5cc5 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:53.503789 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0c911-e1c1-4d14-baea-16a3588d5cc5 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:53.508385 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:53.509278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:53.509591 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:53.509868 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] PATH_INFO: `/v3/users/95c687998a65476c85c467753883da55` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:53.511088 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c687998a65476c85c467753883da55)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:53.513086 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:53.569494 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c687998a65476c85c467753883da55'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:53.582422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c687998a65476c85c467753883da55'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:53.583042 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c687998a65476c85c467753883da55'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:53.591599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c687998a65476c85c467753883da55'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:53.592373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c687998a65476c85c467753883da55'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:53.611243 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1418/2835] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:20:53 2022] DELETE /identity/v3/users/95c687998a65476c85c467753883da55 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:53.616065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca332d-c453-412b-acbf-c3a92da1b3c4 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:53.616365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca332d-c453-412b-acbf-c3a92da1b3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:53.616683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca332d-c453-412b-acbf-c3a92da1b3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:53.635536 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-6eca332d-c453-412b-acbf-c3a92da1b3c4 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:53.636768 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6eca332d-c453-412b-acbf-c3a92da1b3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:53.701264 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1418/2836] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:20:53.711642 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:53.713176 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:53.777202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f62ce7-c5e8-442e-9b06-f0222655392c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:53.780987 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb193965-eff9-4138-939a-e326529ee355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:53.781642 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb193965-eff9-4138-939a-e326529ee355 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:53.781927 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb193965-eff9-4138-939a-e326529ee355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:53.782207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb193965-eff9-4138-939a-e326529ee355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:53.792699 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb193965-eff9-4138-939a-e326529ee355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:53.794154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb193965-eff9-4138-939a-e326529ee355 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:53.804313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1419/2837] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:54.122795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca332d-c453-412b-acbf-c3a92da1b3c4 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:54.124836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-6eca332d-c453-412b-acbf-c3a92da1b3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:54.138911 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca332d-c453-412b-acbf-c3a92da1b3c4 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:54.144108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:54.144998 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:54.145287 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:54.145597 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e admin admin] PATH_INFO: `/v3/projects/379620f3483c484789cde34d81d72220` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:54.146972 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e admin admin] RBAC: Authorizing `identity:delete_project(project_id=379620f3483c484789cde34d81d72220)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:54.148961 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:54.203301 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '379620f3483c484789cde34d81d72220', 'request_id': 'req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:54.211319 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '379620f3483c484789cde34d81d72220', 'request_id': 'req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:54.217738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '379620f3483c484789cde34d81d72220', 'request_id': 'req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:54.231700 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e admin admin] The token cache is being invalidate because project 379620f3483c484789cde34d81d72220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:20:54.231992 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:54.232733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-d3c90bac-f9a6-4d03-a1bf-d8b4d408937e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:20:54.234404 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1419/2838] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:20:54 2022] DELETE /identity/v3/projects/379620f3483c484789cde34d81d72220 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:54.241942 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2859a97-a296-4db7-8333-bd786f411ad9 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:54.242662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2859a97-a296-4db7-8333-bd786f411ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:54.242662 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2859a97-a296-4db7-8333-bd786f411ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:54.264105 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-d2859a97-a296-4db7-8333-bd786f411ad9 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:54.265387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d2859a97-a296-4db7-8333-bd786f411ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:54.287157 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:54.354735 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1420/2839] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:54.363733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f46f1b-aaee-49ee-af9e-bdd631c845b6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:54.364324 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f46f1b-aaee-49ee-af9e-bdd631c845b6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:54.364732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f46f1b-aaee-49ee-af9e-bdd631c845b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:54.396358 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-72f46f1b-aaee-49ee-af9e-bdd631c845b6 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:54.398276 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-72f46f1b-aaee-49ee-af9e-bdd631c845b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:54.408753 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1420/2840] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:54.414271 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2859a97-a296-4db7-8333-bd786f411ad9 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:54.424998 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2859a97-a296-4db7-8333-bd786f411ad9 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:54.430051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eb6bf-570c-485f-abb3-013f27e93f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:54.430738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eb6bf-570c-485f-abb3-013f27e93f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:54.431020 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eb6bf-570c-485f-abb3-013f27e93f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:54.431298 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eb6bf-570c-485f-abb3-013f27e93f44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:54.432342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70eb6bf-570c-485f-abb3-013f27e93f44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:20:54.432653 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70eb6bf-570c-485f-abb3-013f27e93f44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:54.434288 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70eb6bf-570c-485f-abb3-013f27e93f44 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:54.448519 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1421/2841] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:20:54.453606 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f46f1b-aaee-49ee-af9e-bdd631c845b6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:54.465252 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f46f1b-aaee-49ee-af9e-bdd631c845b6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:54.471441 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a30c2f-1eec-4bd7-bb44-b0da6ddd962b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:54.472051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a30c2f-1eec-4bd7-bb44-b0da6ddd962b admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:54.472338 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a30c2f-1eec-4bd7-bb44-b0da6ddd962b admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:54.472630 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a30c2f-1eec-4bd7-bb44-b0da6ddd962b admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:54.473403 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a30c2f-1eec-4bd7-bb44-b0da6ddd962b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:54.476769 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a30c2f-1eec-4bd7-bb44-b0da6ddd962b admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:54.513129 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1421/2842] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:20:54 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:20:54.519109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eb6bf-570c-485f-abb3-013f27e93f44 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:54.537780 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eb6bf-570c-485f-abb3-013f27e93f44 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:54.625081 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef05fb-e3f9-4fdc-a4ae-94e7504d5f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:54.626045 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef05fb-e3f9-4fdc-a4ae-94e7504d5f67 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:54.626455 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef05fb-e3f9-4fdc-a4ae-94e7504d5f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:54.627784 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef05fb-e3f9-4fdc-a4ae-94e7504d5f67 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:54.629372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef05fb-e3f9-4fdc-a4ae-94e7504d5f67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:54.632675 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef05fb-e3f9-4fdc-a4ae-94e7504d5f67 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:54.661985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1422/2843] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:20:54 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:20:54.665732 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a30c2f-1eec-4bd7-bb44-b0da6ddd962b admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:54.677160 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a30c2f-1eec-4bd7-bb44-b0da6ddd962b admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:54.682348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9ed90-9e54-4e60-91d4-cefbbd90bdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:54.683051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9ed90-9e54-4e60-91d4-cefbbd90bdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:54.683339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9ed90-9e54-4e60-91d4-cefbbd90bdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:54.683655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9ed90-9e54-4e60-91d4-cefbbd90bdf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:54.684406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9ed90-9e54-4e60-91d4-cefbbd90bdf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:54.686050 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9ed90-9e54-4e60-91d4-cefbbd90bdf0 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:54.700738 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1422/2844] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:20:54.705994 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef05fb-e3f9-4fdc-a4ae-94e7504d5f67 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:54.717217 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef05fb-e3f9-4fdc-a4ae-94e7504d5f67 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:54.722788 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359faab-420b-4418-b332-7c2c4a2e8bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:54.723025 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359faab-420b-4418-b332-7c2c4a2e8bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:54.723283 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359faab-420b-4418-b332-7c2c4a2e8bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:54.724153 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359faab-420b-4418-b332-7c2c4a2e8bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:54.724499 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359faab-420b-4418-b332-7c2c4a2e8bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:54.726168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359faab-420b-4418-b332-7c2c4a2e8bdf admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:54.739861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1423/2845] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:20:54.745026 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9ed90-9e54-4e60-91d4-cefbbd90bdf0 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:54.756147 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9ed90-9e54-4e60-91d4-cefbbd90bdf0 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:54.763107 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb67cda-79d0-4643-8c1f-8ff9449ba807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:54.764144 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb67cda-79d0-4643-8c1f-8ff9449ba807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:54.764436 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb67cda-79d0-4643-8c1f-8ff9449ba807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:54.764914 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb67cda-79d0-4643-8c1f-8ff9449ba807 admin admin] PATH_INFO: `/v3/projects/67f41dcdf6564bbd910a2285568941a1/users/d942201423da427b848dfd44b33358b8/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:54.783397 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb67cda-79d0-4643-8c1f-8ff9449ba807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67f41dcdf6564bbd910a2285568941a1, user_id=d942201423da427b848dfd44b33358b8, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:54.786215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb67cda-79d0-4643-8c1f-8ff9449ba807 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:54.800938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1423/2846] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:20:54 2022] PUT /identity/v3/projects/67f41dcdf6564bbd910a2285568941a1/users/d942201423da427b848dfd44b33358b8/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:20:54.806333 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2eb65b-d43e-45f1-b9c1-c164a2f77e1d None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:54.806623 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2eb65b-d43e-45f1-b9c1-c164a2f77e1d None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:54.807313 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2eb65b-d43e-45f1-b9c1-c164a2f77e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:54.827454 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-ba2eb65b-d43e-45f1-b9c1-c164a2f77e1d None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:54.829057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ba2eb65b-d43e-45f1-b9c1-c164a2f77e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:54.902275 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1424/2847] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:20:54.912031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb67cda-79d0-4643-8c1f-8ff9449ba807 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:20:54.913790 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-efb67cda-79d0-4643-8c1f-8ff9449ba807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:54.936821 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:54.978667 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb67cda-79d0-4643-8c1f-8ff9449ba807 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:20:54.982959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa46ff7-a693-401d-9ebc-8f700284094e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:20:54.983681 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa46ff7-a693-401d-9ebc-8f700284094e service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:54.983974 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa46ff7-a693-401d-9ebc-8f700284094e service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:54.984260 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa46ff7-a693-401d-9ebc-8f700284094e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:54.994965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa46ff7-a693-401d-9ebc-8f700284094e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:20:54.996501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa46ff7-a693-401d-9ebc-8f700284094e service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:20:55.007916 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1424/2848] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:20:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:20:59.851375 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1e9b7-60a1-4c08-90ad-e87ed2e3ad6f None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:20:59.851938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1e9b7-60a1-4c08-90ad-e87ed2e3ad6f None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:20:59.851938 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1e9b7-60a1-4c08-90ad-e87ed2e3ad6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:20:59.884622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-80a1e9b7-60a1-4c08-90ad-e87ed2e3ad6f None None] MFA Rules not processed for user `d942201423da427b848dfd44b33358b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:20:59.886357 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-80a1e9b7-60a1-4c08-90ad-e87ed2e3ad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:20:59.919581 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:20:59.994282 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1425/2849] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:20:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:20:59.998846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa46ff7-a693-401d-9ebc-8f700284094e service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:00.010172 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa46ff7-a693-401d-9ebc-8f700284094e service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:00.019087 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5d130-ec29-4599-871b-8479e9ac5a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:00.020685 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5d130-ec29-4599-871b-8479e9ac5a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:00.021465 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5d130-ec29-4599-871b-8479e9ac5a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:00.022362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5d130-ec29-4599-871b-8479e9ac5a5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:00.023698 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5d130-ec29-4599-871b-8479e9ac5a5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:00.026233 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5d130-ec29-4599-871b-8479e9ac5a5f admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:00.056225 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1425/2850] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:20:59 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:21:00.060725 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1e9b7-60a1-4c08-90ad-e87ed2e3ad6f None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:00.076589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1e9b7-60a1-4c08-90ad-e87ed2e3ad6f None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:00.154098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0e2ee-d1b3-40d3-b37d-4a0d057de5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:00.154787 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0e2ee-d1b3-40d3-b37d-4a0d057de5da admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:00.155075 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0e2ee-d1b3-40d3-b37d-4a0d057de5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:00.155357 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0e2ee-d1b3-40d3-b37d-4a0d057de5da admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:00.156091 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba0e2ee-d1b3-40d3-b37d-4a0d057de5da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:00.158051 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba0e2ee-d1b3-40d3-b37d-4a0d057de5da admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:00.178440 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1426/2851] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:21:00 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:21:00.182903 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5d130-ec29-4599-871b-8479e9ac5a5f admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:00.194538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5d130-ec29-4599-871b-8479e9ac5a5f admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:00.203016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee815fd-e897-4245-8bda-feb2f881eabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:00.203814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee815fd-e897-4245-8bda-feb2f881eabe admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:00.204106 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee815fd-e897-4245-8bda-feb2f881eabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:00.204391 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee815fd-e897-4245-8bda-feb2f881eabe admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:00.205712 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee815fd-e897-4245-8bda-feb2f881eabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:00.207398 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee815fd-e897-4245-8bda-feb2f881eabe admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:00.218856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1426/2852] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:21:00.223488 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0e2ee-d1b3-40d3-b37d-4a0d057de5da admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:00.233872 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0e2ee-d1b3-40d3-b37d-4a0d057de5da admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:00.239593 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d6a85-f779-4976-838f-4126f0d9b30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:00.240589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d6a85-f779-4976-838f-4126f0d9b30e admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:00.240875 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d6a85-f779-4976-838f-4126f0d9b30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:00.241145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d6a85-f779-4976-838f-4126f0d9b30e admin admin] PATH_INFO: `/v3/projects/b2ef0d38bcd64142b153908b06334091/users/45c0a98b982d444f8aec8107277df32c/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:00.257477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d6a85-f779-4976-838f-4126f0d9b30e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ef0d38bcd64142b153908b06334091, user_id=45c0a98b982d444f8aec8107277df32c, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:00.260095 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d6a85-f779-4976-838f-4126f0d9b30e admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:00.274674 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1427/2853] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:21:00 2022] PUT /identity/v3/projects/b2ef0d38bcd64142b153908b06334091/users/45c0a98b982d444f8aec8107277df32c/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:21:00.278929 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee815fd-e897-4245-8bda-feb2f881eabe admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:00.289550 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee815fd-e897-4245-8bda-feb2f881eabe admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:00.362889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccaf9b-09b2-4e60-9fe1-a6f290e9967c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:00.363733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccaf9b-09b2-4e60-9fe1-a6f290e9967c admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:00.364234 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccaf9b-09b2-4e60-9fe1-a6f290e9967c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:00.364716 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccaf9b-09b2-4e60-9fe1-a6f290e9967c admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:00.365695 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccaf9b-09b2-4e60-9fe1-a6f290e9967c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:00.367463 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccaf9b-09b2-4e60-9fe1-a6f290e9967c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:00.379293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1427/2854] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:21:00.385730 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d6a85-f779-4976-838f-4126f0d9b30e admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:00.400911 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d6a85-f779-4976-838f-4126f0d9b30e admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:00.405894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04703989-32c5-4baa-a33e-fcf9a8860c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:00.406804 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04703989-32c5-4baa-a33e-fcf9a8860c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:00.407087 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04703989-32c5-4baa-a33e-fcf9a8860c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:00.407362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04703989-32c5-4baa-a33e-fcf9a8860c04 admin admin] PATH_INFO: `/v3/projects/b2ef0d38bcd64142b153908b06334091/users/45c0a98b982d444f8aec8107277df32c/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:00.409601 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04703989-32c5-4baa-a33e-fcf9a8860c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ef0d38bcd64142b153908b06334091, user_id=45c0a98b982d444f8aec8107277df32c, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:00.412304 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04703989-32c5-4baa-a33e-fcf9a8860c04 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:00.424799 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1428/2855] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:21:00 2022] PUT /identity/v3/projects/b2ef0d38bcd64142b153908b06334091/users/45c0a98b982d444f8aec8107277df32c/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:21:06.556094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c12ef4-fc64-4b09-8aca-ceb2ca9b5e71 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:06.557851 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c12ef4-fc64-4b09-8aca-ceb2ca9b5e71 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:06.558315 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c12ef4-fc64-4b09-8aca-ceb2ca9b5e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:06.596027 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-79c12ef4-fc64-4b09-8aca-ceb2ca9b5e71 None None] MFA Rules not processed for user `45c0a98b982d444f8aec8107277df32c`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:06.597268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-79c12ef4-fc64-4b09-8aca-ceb2ca9b5e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:06.686986 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1428/2856] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:21:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:06.705317 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04703989-32c5-4baa-a33e-fcf9a8860c04 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:06.707958 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-04703989-32c5-4baa-a33e-fcf9a8860c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:06.820396 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04703989-32c5-4baa-a33e-fcf9a8860c04 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:06.828874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c353d4-3635-402c-acb7-27e73ec14421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:06.830599 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c353d4-3635-402c-acb7-27e73ec14421 service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:06.831117 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c353d4-3635-402c-acb7-27e73ec14421 service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:06.831665 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c353d4-3635-402c-acb7-27e73ec14421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:06.841371 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:21:06.933408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c353d4-3635-402c-acb7-27e73ec14421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:06.935161 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c353d4-3635-402c-acb7-27e73ec14421 service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:06.946563 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1429/2857] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:21:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:21:16.683486 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:21:16.941439 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=24 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:21:43.910614 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0777d6e-d229-4faa-a785-ee36cc6cdc8b None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:43.911179 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0777d6e-d229-4faa-a785-ee36cc6cdc8b None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:43.911355 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0777d6e-d229-4faa-a785-ee36cc6cdc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:43.931031 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d0777d6e-d229-4faa-a785-ee36cc6cdc8b None None] MFA Rules not processed for user `d942201423da427b848dfd44b33358b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:21:43.932265 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d0777d6e-d229-4faa-a785-ee36cc6cdc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:43.940089 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1429/2858] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:21:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:21:44.271034 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c353d4-3635-402c-acb7-27e73ec14421 service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:21:44.328222 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c353d4-3635-402c-acb7-27e73ec14421 service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:21:44.331861 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af0263-eb8e-48b1-ade1-ae409209cfad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:21:44.332515 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af0263-eb8e-48b1-ade1-ae409209cfad service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:21:44.332798 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af0263-eb8e-48b1-ade1-ae409209cfad service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:21:44.333078 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af0263-eb8e-48b1-ade1-ae409209cfad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:21:44.334867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-22af0263-eb8e-48b1-ade1-ae409209cfad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:21:44.345155 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af0263-eb8e-48b1-ade1-ae409209cfad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:21:44.346600 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af0263-eb8e-48b1-ade1-ae409209cfad service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:21:44.355792 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1430/2859] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:21:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:21:53.925963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:21:54.352307 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=26 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:00.452421 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704cbae-8598-4099-8631-57b89a1aa4d0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:00.452888 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704cbae-8598-4099-8631-57b89a1aa4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:00.452983 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704cbae-8598-4099-8631-57b89a1aa4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:00.471545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d704cbae-8598-4099-8631-57b89a1aa4d0 None None] MFA Rules not processed for user `6d9987a8042840caa38d2112fcce404a`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:22:00.472756 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d704cbae-8598-4099-8631-57b89a1aa4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:00.531786 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1430/2860] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:22:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:22:08.627372 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af0263-eb8e-48b1-ade1-ae409209cfad service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:08.637284 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af0263-eb8e-48b1-ade1-ae409209cfad service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:08.641308 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b42096-26fc-4f0f-9842-a380e45ecb3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:08.641939 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b42096-26fc-4f0f-9842-a380e45ecb3b service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:08.642223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b42096-26fc-4f0f-9842-a380e45ecb3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:08.642511 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b42096-26fc-4f0f-9842-a380e45ecb3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:08.652259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b42096-26fc-4f0f-9842-a380e45ecb3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:08.653679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b42096-26fc-4f0f-9842-a380e45ecb3b service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:08.663230 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1431/2861] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:10.522348 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=25 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:18.658891 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=6 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:38.248816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704cbae-8598-4099-8631-57b89a1aa4d0 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:38.258612 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704cbae-8598-4099-8631-57b89a1aa4d0 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:38.262190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698bb42-cf4e-4078-adf3-52ca24dbf05e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:38.262847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698bb42-cf4e-4078-adf3-52ca24dbf05e service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:38.263131 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698bb42-cf4e-4078-adf3-52ca24dbf05e service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:38.263428 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698bb42-cf4e-4078-adf3-52ca24dbf05e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:38.265169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8698bb42-cf4e-4078-adf3-52ca24dbf05e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:38.278547 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698bb42-cf4e-4078-adf3-52ca24dbf05e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:38.280000 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698bb42-cf4e-4078-adf3-52ca24dbf05e service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:38.291406 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1431/2862] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:39.469390 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b42096-26fc-4f0f-9842-a380e45ecb3b service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:22:39.470937 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d7b42096-26fc-4f0f-9842-a380e45ecb3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:39.483956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b42096-26fc-4f0f-9842-a380e45ecb3b service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:22:39.487684 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf496607-27ee-4eb7-916f-04e43fb44071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:22:39.488338 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf496607-27ee-4eb7-916f-04e43fb44071 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:22:39.488622 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf496607-27ee-4eb7-916f-04e43fb44071 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:22:39.488926 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf496607-27ee-4eb7-916f-04e43fb44071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:22:39.490769 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-cf496607-27ee-4eb7-916f-04e43fb44071 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:22:39.515076 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf496607-27ee-4eb7-916f-04e43fb44071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:22:39.517259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf496607-27ee-4eb7-916f-04e43fb44071 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:22:39.541197 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1432/2863] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:22:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:22:48.286055 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=6 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:22:49.528168 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=6 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:05.487573 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698bb42-cf4e-4078-adf3-52ca24dbf05e service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:05.489056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-8698bb42-cf4e-4078-adf3-52ca24dbf05e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:05.572329 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698bb42-cf4e-4078-adf3-52ca24dbf05e service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:05.576928 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515181c-f792-4d2c-b4df-2473a46a58d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:05.577830 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515181c-f792-4d2c-b4df-2473a46a58d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:05.578123 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515181c-f792-4d2c-b4df-2473a46a58d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:05.578405 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515181c-f792-4d2c-b4df-2473a46a58d1 admin admin] PATH_INFO: `/v3/users/6d9987a8042840caa38d2112fcce404a` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:05.579740 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515181c-f792-4d2c-b4df-2473a46a58d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d9987a8042840caa38d2112fcce404a)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:05.581714 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515181c-f792-4d2c-b4df-2473a46a58d1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:05.634795 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6515181c-f792-4d2c-b4df-2473a46a58d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d9987a8042840caa38d2112fcce404a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:05.635506 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6515181c-f792-4d2c-b4df-2473a46a58d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d9987a8042840caa38d2112fcce404a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:05.642600 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6515181c-f792-4d2c-b4df-2473a46a58d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d9987a8042840caa38d2112fcce404a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:05.650443 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6515181c-f792-4d2c-b4df-2473a46a58d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d9987a8042840caa38d2112fcce404a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:05.651130 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-6515181c-f792-4d2c-b4df-2473a46a58d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d9987a8042840caa38d2112fcce404a'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:05.670137 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1432/2864] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:23:05 2022] DELETE /identity/v3/users/6d9987a8042840caa38d2112fcce404a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:05.851788 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf496607-27ee-4eb7-916f-04e43fb44071 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:05.866286 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf496607-27ee-4eb7-916f-04e43fb44071 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:05.948846 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:05.949647 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:05.949932 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:05.950219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] PATH_INFO: `/v3/projects/183612235b834368be67c02434f45d36` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:05.951529 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=183612235b834368be67c02434f45d36)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:05.953528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:06.008365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '183612235b834368be67c02434f45d36', 'request_id': 'req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:06.015035 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '183612235b834368be67c02434f45d36', 'request_id': 'req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:06.021017 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '183612235b834368be67c02434f45d36', 'request_id': 'req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:06.036061 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] The token cache is being invalidate because project 183612235b834368be67c02434f45d36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:23:06.036365 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:06.037006 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:06.038596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1433/2865] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:23:05 2022] DELETE /identity/v3/projects/183612235b834368be67c02434f45d36 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:15.666476 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:16.032400 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:18.236232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8e052-b407-4930-96a5-16b8b38b504a None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:18.237121 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8e052-b407-4930-96a5-16b8b38b504a None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:18.237361 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8e052-b407-4930-96a5-16b8b38b504a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:18.257608 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-75f8e052-b407-4930-96a5-16b8b38b504a None None] MFA Rules not processed for user `f40c0fcc93cf4bbf8d35c1de70d087ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:18.258856 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-75f8e052-b407-4930-96a5-16b8b38b504a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:18.320963 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1433/2866] 10.222.0.75 () {58 vars in 967 bytes} [Thu Jul 14 11:23:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:23:18.371044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:18.372556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:18.444612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b213e-a9f5-49ff-a58e-8b8d7eab15b8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:18.449766 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745bdb3-5b5e-475a-9922-e0d99c4ce2ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '978e71dfbb134f41871114824b03fbeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:18.450413 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745bdb3-5b5e-475a-9922-e0d99c4ce2ea service swift] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:18.450724 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745bdb3-5b5e-475a-9922-e0d99c4ce2ea service swift] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:18.451001 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745bdb3-5b5e-475a-9922-e0d99c4ce2ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:18.459746 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745bdb3-5b5e-475a-9922-e0d99c4ce2ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:18.461347 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745bdb3-5b5e-475a-9922-e0d99c4ce2ea service swift] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:18.469825 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1434/2867] 10.222.0.75 () {60 vars in 1384 bytes} [Thu Jul 14 11:23:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:23:21.991017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8e052-b407-4930-96a5-16b8b38b504a None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:21.992576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-75f8e052-b407-4930-96a5-16b8b38b504a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:22.107979 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8e052-b407-4930-96a5-16b8b38b504a None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:22.123730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee64503-7a05-42e8-b320-3eed7984c78b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:22.123730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee64503-7a05-42e8-b320-3eed7984c78b service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:22.123730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee64503-7a05-42e8-b320-3eed7984c78b service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:22.123730 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee64503-7a05-42e8-b320-3eed7984c78b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:22.124965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-bee64503-7a05-42e8-b320-3eed7984c78b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:22.131226 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee64503-7a05-42e8-b320-3eed7984c78b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:22.133042 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee64503-7a05-42e8-b320-3eed7984c78b service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:22.145068 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1434/2868] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:23:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:23:25.998840 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745bdb3-5b5e-475a-9922-e0d99c4ce2ea service swift] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:26.001842 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-d745bdb3-5b5e-475a-9922-e0d99c4ce2ea service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:26.084340 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745bdb3-5b5e-475a-9922-e0d99c4ce2ea service swift] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:26.090706 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:26.091556 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:26.091839 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:26.092122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] PATH_INFO: `/v3/users/677282eb944e45d0ae92991660551c4e` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:26.093316 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=677282eb944e45d0ae92991660551c4e)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:26.095269 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:26.143533 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677282eb944e45d0ae92991660551c4e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.152826 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677282eb944e45d0ae92991660551c4e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.153520 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677282eb944e45d0ae92991660551c4e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.160863 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677282eb944e45d0ae92991660551c4e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.161534 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677282eb944e45d0ae92991660551c4e'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.180624 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1435/2869] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:23:25 2022] DELETE /identity/v3/users/677282eb944e45d0ae92991660551c4e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:26.372331 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee64503-7a05-42e8-b320-3eed7984c78b service placement] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:26.383454 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee64503-7a05-42e8-b320-3eed7984c78b service placement] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:26.441736 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:26.442607 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:26.442889 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:26.443167 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] PATH_INFO: `/v3/projects/9b0b1b94221f442daa87ea1e3ca55df3` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:26.444572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b0b1b94221f442daa87ea1e3ca55df3)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:26.446282 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:26.483073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:26.492143 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b0b1b94221f442daa87ea1e3ca55df3', 'request_id': 'req-3e115d02-aac8-4460-bc88-85ff1fb5c131'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.498926 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b0b1b94221f442daa87ea1e3ca55df3', 'request_id': 'req-3e115d02-aac8-4460-bc88-85ff1fb5c131'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.504652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b0b1b94221f442daa87ea1e3ca55df3', 'request_id': 'req-3e115d02-aac8-4460-bc88-85ff1fb5c131'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.518039 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] The token cache is being invalidate because project 9b0b1b94221f442daa87ea1e3ca55df3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:23:26.518341 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.518974 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.520572 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1435/2870] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:23:26 2022] DELETE /identity/v3/projects/9b0b1b94221f442daa87ea1e3ca55df3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:26.524771 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:26.526199 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:26.539521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb0882-a0eb-4d8c-ab67-20f99d6518b0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:26.544116 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:26.597555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847b49f-e265-4339-aac6-424290cade20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:26.598427 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847b49f-e265-4339-aac6-424290cade20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:26.598714 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847b49f-e265-4339-aac6-424290cade20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:26.598997 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847b49f-e265-4339-aac6-424290cade20 admin admin] PATH_INFO: `/v3/users/feb53ffd683245b7beb4034ba2b260a7` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:26.600226 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847b49f-e265-4339-aac6-424290cade20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb53ffd683245b7beb4034ba2b260a7)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:26.602154 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847b49f-e265-4339-aac6-424290cade20 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:26.647941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d847b49f-e265-4339-aac6-424290cade20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb53ffd683245b7beb4034ba2b260a7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.655278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d847b49f-e265-4339-aac6-424290cade20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb53ffd683245b7beb4034ba2b260a7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.655921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d847b49f-e265-4339-aac6-424290cade20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb53ffd683245b7beb4034ba2b260a7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.662797 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d847b49f-e265-4339-aac6-424290cade20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb53ffd683245b7beb4034ba2b260a7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.663460 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-d847b49f-e265-4339-aac6-424290cade20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb53ffd683245b7beb4034ba2b260a7'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:26.681910 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1436/2871] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:23:26 2022] DELETE /identity/v3/users/feb53ffd683245b7beb4034ba2b260a7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:26.857588 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:26.859119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:26.883362 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e115d02-aac8-4460-bc88-85ff1fb5c131 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:26.949672 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ff73a-5955-49e5-a165-10bcb2599587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:26.950555 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ff73a-5955-49e5-a165-10bcb2599587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:26.950839 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ff73a-5955-49e5-a165-10bcb2599587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:26.951119 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ff73a-5955-49e5-a165-10bcb2599587 admin admin] PATH_INFO: `/v3/projects/c8e814c52ec14d749384e4793e146534` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:26.952336 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ff73a-5955-49e5-a165-10bcb2599587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e814c52ec14d749384e4793e146534)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:26.954259 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ff73a-5955-49e5-a165-10bcb2599587 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:27.003715 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3f4ff73a-5955-49e5-a165-10bcb2599587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e814c52ec14d749384e4793e146534', 'request_id': 'req-3f4ff73a-5955-49e5-a165-10bcb2599587'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:27.012153 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3f4ff73a-5955-49e5-a165-10bcb2599587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e814c52ec14d749384e4793e146534', 'request_id': 'req-3f4ff73a-5955-49e5-a165-10bcb2599587'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:27.018267 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3f4ff73a-5955-49e5-a165-10bcb2599587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e814c52ec14d749384e4793e146534', 'request_id': 'req-3f4ff73a-5955-49e5-a165-10bcb2599587'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:27.034469 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3f4ff73a-5955-49e5-a165-10bcb2599587 admin admin] The token cache is being invalidate because project c8e814c52ec14d749384e4793e146534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:23:27.034770 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3f4ff73a-5955-49e5-a165-10bcb2599587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:27.035908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-3f4ff73a-5955-49e5-a165-10bcb2599587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:23:27.037749 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1436/2872] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:23:26 2022] DELETE /identity/v3/projects/c8e814c52ec14d749384e4793e146534 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:27.043299 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811e5bb-dd0c-40e0-92e6-b3fc75463d45 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.043821 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811e5bb-dd0c-40e0-92e6-b3fc75463d45 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.043945 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811e5bb-dd0c-40e0-92e6-b3fc75463d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.062781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9811e5bb-dd0c-40e0-92e6-b3fc75463d45 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:27.064038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9811e5bb-dd0c-40e0-92e6-b3fc75463d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:27.131145 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1437/2873] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:23:27.139363 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3d064-4789-4434-9e20-7cc849685244 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.139683 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3d064-4789-4434-9e20-7cc849685244 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.139968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3d064-4789-4434-9e20-7cc849685244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.159864 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-d2a3d064-4789-4434-9e20-7cc849685244 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:27.161103 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-d2a3d064-4789-4434-9e20-7cc849685244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:27.169956 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1437/2874] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:23:27.174401 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811e5bb-dd0c-40e0-92e6-b3fc75463d45 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:27.184650 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811e5bb-dd0c-40e0-92e6-b3fc75463d45 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:27.189396 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5306a-b5b1-4228-acbb-7eb765b4b2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:27.190110 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5306a-b5b1-4228-acbb-7eb765b4b2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.190395 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5306a-b5b1-4228-acbb-7eb765b4b2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.190700 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5306a-b5b1-4228-acbb-7eb765b4b2d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.191482 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5306a-b5b1-4228-acbb-7eb765b4b2d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80182) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:23:27.191795 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5306a-b5b1-4228-acbb-7eb765b4b2d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:27.193441 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5306a-b5b1-4228-acbb-7eb765b4b2d0 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:27.207259 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1438/2875] 10.222.0.75 () {58 vars in 1152 bytes} [Thu Jul 14 11:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:23:27.213269 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3d064-4789-4434-9e20-7cc849685244 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:27.223003 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3d064-4789-4434-9e20-7cc849685244 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:27.228720 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48c27b-aebf-4ac1-b8d3-6d228532292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:27.229390 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48c27b-aebf-4ac1-b8d3-6d228532292a admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.229677 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48c27b-aebf-4ac1-b8d3-6d228532292a admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.229959 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d48c27b-aebf-4ac1-b8d3-6d228532292a admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.230666 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d48c27b-aebf-4ac1-b8d3-6d228532292a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:27.232649 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d48c27b-aebf-4ac1-b8d3-6d228532292a admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:27.258647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1438/2876] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:23:27 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:27.262983 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5306a-b5b1-4228-acbb-7eb765b4b2d0 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:27.273069 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5306a-b5b1-4228-acbb-7eb765b4b2d0 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:27.286115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:27.335612 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b8e43-c9fb-4159-8892-c556bc926233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:27.336290 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b8e43-c9fb-4159-8892-c556bc926233 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.336576 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b8e43-c9fb-4159-8892-c556bc926233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.336856 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b8e43-c9fb-4159-8892-c556bc926233 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.338038 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b8e43-c9fb-4159-8892-c556bc926233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:27.339985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b8e43-c9fb-4159-8892-c556bc926233 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:27.359587 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1439/2877] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:23:27 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:27.363843 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48c27b-aebf-4ac1-b8d3-6d228532292a admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:27.374108 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d48c27b-aebf-4ac1-b8d3-6d228532292a admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:27.378827 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ed61e-4612-4f3b-a1fd-e3ee8b239392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:27.379766 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ed61e-4612-4f3b-a1fd-e3ee8b239392 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.380058 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ed61e-4612-4f3b-a1fd-e3ee8b239392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.380376 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ed61e-4612-4f3b-a1fd-e3ee8b239392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.381070 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ed61e-4612-4f3b-a1fd-e3ee8b239392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:27.382816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ed61e-4612-4f3b-a1fd-e3ee8b239392 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:27.394367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1439/2878] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:23:27.398342 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b8e43-c9fb-4159-8892-c556bc926233 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:27.408865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b8e43-c9fb-4159-8892-c556bc926233 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:27.413449 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b6c6f-91ca-4aa0-bb77-fe03dd0bae3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:27.414106 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b6c6f-91ca-4aa0-bb77-fe03dd0bae3b admin admin] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.414383 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b6c6f-91ca-4aa0-bb77-fe03dd0bae3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.414660 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b6c6f-91ca-4aa0-bb77-fe03dd0bae3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.416070 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b6c6f-91ca-4aa0-bb77-fe03dd0bae3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:27.417668 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b6c6f-91ca-4aa0-bb77-fe03dd0bae3b admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:27.427812 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1440/2879] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:23:27.431875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ed61e-4612-4f3b-a1fd-e3ee8b239392 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:27.442472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ed61e-4612-4f3b-a1fd-e3ee8b239392 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:27.447578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317057cb-da90-4d15-ace2-4bcb2b1e9e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:27.448453 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317057cb-da90-4d15-ace2-4bcb2b1e9e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.448735 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317057cb-da90-4d15-ace2-4bcb2b1e9e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.449017 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317057cb-da90-4d15-ace2-4bcb2b1e9e48 admin admin] PATH_INFO: `/v3/projects/67f0b0b647c442b9a682fd37d756400a/users/65c3d05f3b904c0f999984dda155f1cf/roles/9a562f9a0cad4304852f3183eef398d2` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.460733 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:27.464571 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317057cb-da90-4d15-ace2-4bcb2b1e9e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67f0b0b647c442b9a682fd37d756400a, user_id=65c3d05f3b904c0f999984dda155f1cf, role_id=9a562f9a0cad4304852f3183eef398d2)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:27.467806 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317057cb-da90-4d15-ace2-4bcb2b1e9e48 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:27.482909 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1440/2880] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:23:27 2022] PUT /identity/v3/projects/67f0b0b647c442b9a682fd37d756400a/users/65c3d05f3b904c0f999984dda155f1cf/roles/9a562f9a0cad4304852f3183eef398d2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:27.488140 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c116288-9665-439f-aea2-90ed958c0cf5 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.488431 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c116288-9665-439f-aea2-90ed958c0cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.488710 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c116288-9665-439f-aea2-90ed958c0cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.506305 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8c116288-9665-439f-aea2-90ed958c0cf5 None None] MFA Rules not processed for user `eae05ea06648449e9eda17d7c9a3f3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:27.507552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8c116288-9665-439f-aea2-90ed958c0cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:27.569185 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1441/2881] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:23:27.579566 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317057cb-da90-4d15-ace2-4bcb2b1e9e48 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:27.581293 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-317057cb-da90-4d15-ace2-4bcb2b1e9e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:27.638180 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317057cb-da90-4d15-ace2-4bcb2b1e9e48 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:27.641846 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09412100-f606-4bbb-b9ff-a6d4f64e0f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:27.642488 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09412100-f606-4bbb-b9ff-a6d4f64e0f92 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:27.642774 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09412100-f606-4bbb-b9ff-a6d4f64e0f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:27.643048 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09412100-f606-4bbb-b9ff-a6d4f64e0f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:27.652878 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09412100-f606-4bbb-b9ff-a6d4f64e0f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:27.654327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09412100-f606-4bbb-b9ff-a6d4f64e0f92 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:27.664078 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1441/2882] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:23:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:23:32.479785 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03f352-a214-49fe-b23f-cf48bb8536a3 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.480303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03f352-a214-49fe-b23f-cf48bb8536a3 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.480457 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03f352-a214-49fe-b23f-cf48bb8536a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.509309 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-9f03f352-a214-49fe-b23f-cf48bb8536a3 None None] MFA Rules not processed for user `65c3d05f3b904c0f999984dda155f1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:32.510809 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9f03f352-a214-49fe-b23f-cf48bb8536a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:32.582194 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1442/2883] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:23:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:23:32.586755 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09412100-f606-4bbb-b9ff-a6d4f64e0f92 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.597599 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09412100-f606-4bbb-b9ff-a6d4f64e0f92 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.602244 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe523fa-72c6-4c80-a13d-3a1833aa29e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:32.603771 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe523fa-72c6-4c80-a13d-3a1833aa29e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.603965 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe523fa-72c6-4c80-a13d-3a1833aa29e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.604262 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe523fa-72c6-4c80-a13d-3a1833aa29e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.604968 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe523fa-72c6-4c80-a13d-3a1833aa29e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:32.606907 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe523fa-72c6-4c80-a13d-3a1833aa29e1 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:32.634340 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1442/2884] 10.222.0.75 () {60 vars in 1152 bytes} [Thu Jul 14 11:23:32 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:32.638321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03f352-a214-49fe-b23f-cf48bb8536a3 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.649401 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03f352-a214-49fe-b23f-cf48bb8536a3 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.670890 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:32.710464 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c384a9-c6d3-4665-8aaa-d2bb840e5592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:32.711202 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c384a9-c6d3-4665-8aaa-d2bb840e5592 admin admin] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.711443 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c384a9-c6d3-4665-8aaa-d2bb840e5592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.711729 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c384a9-c6d3-4665-8aaa-d2bb840e5592 admin admin] PATH_INFO: `/v3/users` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.712990 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c384a9-c6d3-4665-8aaa-d2bb840e5592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:32.715098 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c384a9-c6d3-4665-8aaa-d2bb840e5592 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:32.745718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1443/2885] 10.222.0.75 () {60 vars in 1143 bytes} [Thu Jul 14 11:23:32 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:23:32.750462 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe523fa-72c6-4c80-a13d-3a1833aa29e1 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.762267 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe523fa-72c6-4c80-a13d-3a1833aa29e1 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.767049 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129e38f-3e41-4f55-adf2-0e9374f30a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:32.767756 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129e38f-3e41-4f55-adf2-0e9374f30a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.768043 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129e38f-3e41-4f55-adf2-0e9374f30a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.768330 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129e38f-3e41-4f55-adf2-0e9374f30a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.769063 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129e38f-3e41-4f55-adf2-0e9374f30a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:32.770693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129e38f-3e41-4f55-adf2-0e9374f30a03 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:32.782223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1443/2886] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:23:32.786979 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c384a9-c6d3-4665-8aaa-d2bb840e5592 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.797300 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c384a9-c6d3-4665-8aaa-d2bb840e5592 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.801931 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128039bd-c896-47cc-a42b-1eff9b5f6f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:32.802808 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128039bd-c896-47cc-a42b-1eff9b5f6f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.803175 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128039bd-c896-47cc-a42b-1eff9b5f6f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.803352 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128039bd-c896-47cc-a42b-1eff9b5f6f22 admin admin] PATH_INFO: `/v3/projects/b9459e6ccd784ecfb7f268823adc0b84/users/d1cdd12de4cc4c19b0182695935a3135/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.818278 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128039bd-c896-47cc-a42b-1eff9b5f6f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9459e6ccd784ecfb7f268823adc0b84, user_id=d1cdd12de4cc4c19b0182695935a3135, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:32.820954 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128039bd-c896-47cc-a42b-1eff9b5f6f22 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:32.834385 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1444/2887] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:23:32 2022] PUT /identity/v3/projects/b9459e6ccd784ecfb7f268823adc0b84/users/d1cdd12de4cc4c19b0182695935a3135/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:32.838933 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129e38f-3e41-4f55-adf2-0e9374f30a03 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.848973 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129e38f-3e41-4f55-adf2-0e9374f30a03 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.905576 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15e9cb-8861-4538-a9ca-5c22e807cfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:32.906249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15e9cb-8861-4538-a9ca-5c22e807cfc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.906538 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15e9cb-8861-4538-a9ca-5c22e807cfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.906818 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15e9cb-8861-4538-a9ca-5c22e807cfc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.907578 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15e9cb-8861-4538-a9ca-5c22e807cfc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:32.909184 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15e9cb-8861-4538-a9ca-5c22e807cfc6 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:32.919169 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1444/2888] 10.222.0.75 () {58 vars in 1121 bytes} [Thu Jul 14 11:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:23:32.923314 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128039bd-c896-47cc-a42b-1eff9b5f6f22 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:32.934277 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128039bd-c896-47cc-a42b-1eff9b5f6f22 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:32.939252 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220525d-d7a1-4dee-a9e5-36c73cefa10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:32.940211 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220525d-d7a1-4dee-a9e5-36c73cefa10c admin admin] REQUEST_METHOD: `PUT` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:32.940498 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220525d-d7a1-4dee-a9e5-36c73cefa10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:32.940905 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220525d-d7a1-4dee-a9e5-36c73cefa10c admin admin] PATH_INFO: `/v3/projects/b9459e6ccd784ecfb7f268823adc0b84/users/d1cdd12de4cc4c19b0182695935a3135/roles/a0a5d694236542a5826b5a4bc33e1932` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:32.943773 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220525d-d7a1-4dee-a9e5-36c73cefa10c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9459e6ccd784ecfb7f268823adc0b84, user_id=d1cdd12de4cc4c19b0182695935a3135, role_id=a0a5d694236542a5826b5a4bc33e1932)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:32.946387 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220525d-d7a1-4dee-a9e5-36c73cefa10c admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:32.961589 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1445/2889] 10.222.0.75 () {60 vars in 1482 bytes} [Thu Jul 14 11:23:32 2022] PUT /identity/v3/projects/b9459e6ccd784ecfb7f268823adc0b84/users/d1cdd12de4cc4c19b0182695935a3135/roles/a0a5d694236542a5826b5a4bc33e1932 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:23:37.766412 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95702b30-9d97-474a-b334-ac23b05379f0 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:37.766915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95702b30-9d97-474a-b334-ac23b05379f0 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:37.767181 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95702b30-9d97-474a-b334-ac23b05379f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:37.799658 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-95702b30-9d97-474a-b334-ac23b05379f0 None None] MFA Rules not processed for user `d1cdd12de4cc4c19b0182695935a3135`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:37.800497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-95702b30-9d97-474a-b334-ac23b05379f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:37.854455 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:37.887475 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1445/2890] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:23:37 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:23:37.904526 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220525d-d7a1-4dee-a9e5-36c73cefa10c admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:23:37.906894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a220525d-d7a1-4dee-a9e5-36c73cefa10c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:37.992261 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220525d-d7a1-4dee-a9e5-36c73cefa10c admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:23:37.998059 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e42a3-86c9-4c00-b211-245ec851cedd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:23:37.998649 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e42a3-86c9-4c00-b211-245ec851cedd service nova] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:37.998964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e42a3-86c9-4c00-b211-245ec851cedd service nova] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:37.999245 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e42a3-86c9-4c00-b211-245ec851cedd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:38.064122 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e42a3-86c9-4c00-b211-245ec851cedd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:23:38.065744 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e42a3-86c9-4c00-b211-245ec851cedd service nova] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:23:38.076472 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1446/2891] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:23:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:23:40.164668 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a38eb3-510f-445e-a98c-7790e354e0b6 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:23:40.165915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a38eb3-510f-445e-a98c-7790e354e0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:23:40.165915 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a38eb3-510f-445e-a98c-7790e354e0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:23:40.185113 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-f8a38eb3-510f-445e-a98c-7790e354e0b6 None None] MFA Rules not processed for user `65c3d05f3b904c0f999984dda155f1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:23:40.186337 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f8a38eb3-510f-445e-a98c-7790e354e0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:23:40.201838 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1446/2892] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:23:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:23:48.071052 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:23:50.178626 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=11 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:16.867398 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e42a3-86c9-4c00-b211-245ec851cedd service nova] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:16.936094 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e42a3-86c9-4c00-b211-245ec851cedd service nova] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:16.940552 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82013b-9202-440e-86b7-f9f7e5d81dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:16.941233 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82013b-9202-440e-86b7-f9f7e5d81dbf service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:16.941521 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82013b-9202-440e-86b7-f9f7e5d81dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:16.941802 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82013b-9202-440e-86b7-f9f7e5d81dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:16.954738 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82013b-9202-440e-86b7-f9f7e5d81dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:16.956373 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82013b-9202-440e-86b7-f9f7e5d81dbf service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:16.971318 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1447/2893] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:24:16 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:24:26.963093 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=26 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:24:50.261995 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a38eb3-510f-445e-a98c-7790e354e0b6 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:50.275750 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a38eb3-510f-445e-a98c-7790e354e0b6 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:50.288563 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ec84-d2d0-4930-97c1-b72976322b64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:50.289248 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ec84-d2d0-4930-97c1-b72976322b64 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:50.289615 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ec84-d2d0-4930-97c1-b72976322b64 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:50.289842 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ec84-d2d0-4930-97c1-b72976322b64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:50.291748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f9f0ec84-d2d0-4930-97c1-b72976322b64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:50.306823 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0ec84-d2d0-4930-97c1-b72976322b64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:50.308491 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0ec84-d2d0-4930-97c1-b72976322b64 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:50.320577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1447/2894] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:24:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:24:51.090658 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9789b-4e26-4413-b2b4-460f3c29bb40 None None] REQUEST_METHOD: `POST` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:51.091182 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9789b-4e26-4413-b2b4-460f3c29bb40 None None] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:51.091297 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9789b-4e26-4413-b2b4-460f3c29bb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:51.109679 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.auth.core [None req-8ca9789b-4e26-4413-b2b4-460f3c29bb40 None None] MFA Rules not processed for user `d942201423da427b848dfd44b33358b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80182) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:24:51.110941 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-8ca9789b-4e26-4413-b2b4-460f3c29bb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:51.175877 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1448/2895] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:24:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:24:51.927099 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ec84-d2d0-4930-97c1-b72976322b64 service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:51.928496 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-f9f0ec84-d2d0-4930-97c1-b72976322b64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:51.999802 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ec84-d2d0-4930-97c1-b72976322b64 service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:52.004822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a28d09-0bba-4a94-9a7d-359d02f8288a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:52.005472 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a28d09-0bba-4a94-9a7d-359d02f8288a service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:52.005762 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a28d09-0bba-4a94-9a7d-359d02f8288a service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:52.006083 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a28d09-0bba-4a94-9a7d-359d02f8288a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:52.007887 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-57a28d09-0bba-4a94-9a7d-359d02f8288a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:52.019174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a28d09-0bba-4a94-9a7d-359d02f8288a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:52.020811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a28d09-0bba-4a94-9a7d-359d02f8288a service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:52.035865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1448/2896] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:24:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:24:53.754423 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9789b-4e26-4413-b2b4-460f3c29bb40 None None] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:24:53.764974 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9789b-4e26-4413-b2b4-460f3c29bb40 None None] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:24:53.768940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816c232-73e7-4ca0-ab29-d01dc3783574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:24:53.769629 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816c232-73e7-4ca0-ab29-d01dc3783574 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:24:53.769912 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816c232-73e7-4ca0-ab29-d01dc3783574 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:24:53.770189 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816c232-73e7-4ca0-ab29-d01dc3783574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:24:53.772843 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5816c232-73e7-4ca0-ab29-d01dc3783574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:24:53.784068 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816c232-73e7-4ca0-ab29-d01dc3783574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:24:53.786147 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816c232-73e7-4ca0-ab29-d01dc3783574 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:24:53.799742 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1449/2897] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:24:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:25:02.030652 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=38 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:03.791940 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=31 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:25:04.673357 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a28d09-0bba-4a94-9a7d-359d02f8288a service placement] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:25:04.674811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-57a28d09-0bba-4a94-9a7d-359d02f8288a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:04.691424 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a28d09-0bba-4a94-9a7d-359d02f8288a service placement] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:25:04.697487 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccda494-0242-4e6f-8ebd-cd42983a2e4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:25:04.698133 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccda494-0242-4e6f-8ebd-cd42983a2e4f service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:25:04.698416 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccda494-0242-4e6f-8ebd-cd42983a2e4f service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:25:04.698696 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccda494-0242-4e6f-8ebd-cd42983a2e4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:25:04.700492 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-2ccda494-0242-4e6f-8ebd-cd42983a2e4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:25:04.716183 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccda494-0242-4e6f-8ebd-cd42983a2e4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:25:04.718006 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccda494-0242-4e6f-8ebd-cd42983a2e4f service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:25:04.730270 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1449/2898] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:25:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:25:14.724214 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=6 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:33.566625 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816c232-73e7-4ca0-ab29-d01dc3783574 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:33.568594 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5816c232-73e7-4ca0-ab29-d01dc3783574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:33.662953 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816c232-73e7-4ca0-ab29-d01dc3783574 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:33.668214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf59a88-e1f7-4ca6-af4e-acd8cbdddad3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:33.668865 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf59a88-e1f7-4ca6-af4e-acd8cbdddad3 service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:33.669141 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf59a88-e1f7-4ca6-af4e-acd8cbdddad3 service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:33.669422 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf59a88-e1f7-4ca6-af4e-acd8cbdddad3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:33.672303 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-edf59a88-e1f7-4ca6-af4e-acd8cbdddad3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:33.684658 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf59a88-e1f7-4ca6-af4e-acd8cbdddad3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:33.686522 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf59a88-e1f7-4ca6-af4e-acd8cbdddad3 service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:33.697379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1450/2899] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:43.692749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=37 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:47.521044 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccda494-0242-4e6f-8ebd-cd42983a2e4f service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:47.558982 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccda494-0242-4e6f-8ebd-cd42983a2e4f service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:47.562941 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eed49e-2f23-4829-bd48-f4c671d49af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:47.563620 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eed49e-2f23-4829-bd48-f4c671d49af9 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:47.563908 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eed49e-2f23-4829-bd48-f4c671d49af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:47.564190 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eed49e-2f23-4829-bd48-f4c671d49af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:47.566001 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-48eed49e-2f23-4829-bd48-f4c671d49af9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:47.590693 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eed49e-2f23-4829-bd48-f4c671d49af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:47.592174 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eed49e-2f23-4829-bd48-f4c671d49af9 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:47.600962 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1450/2900] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:47.695060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf59a88-e1f7-4ca6-af4e-acd8cbdddad3 service placement] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:47.696655 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-edf59a88-e1f7-4ca6-af4e-acd8cbdddad3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:47.709044 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf59a88-e1f7-4ca6-af4e-acd8cbdddad3 service placement] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:47.712687 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee418f-7238-4d91-b7f0-b1c60ddcdf1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:47.713358 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee418f-7238-4d91-b7f0-b1c60ddcdf1c service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:47.713643 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee418f-7238-4d91-b7f0-b1c60ddcdf1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:47.713925 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee418f-7238-4d91-b7f0-b1c60ddcdf1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:47.715739 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-acee418f-7238-4d91-b7f0-b1c60ddcdf1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:47.726639 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee418f-7238-4d91-b7f0-b1c60ddcdf1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:47.728115 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee418f-7238-4d91-b7f0-b1c60ddcdf1c service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:47.736537 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1451/2901] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:47.848900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eed49e-2f23-4829-bd48-f4c671d49af9 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:26:47.858854 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eed49e-2f23-4829-bd48-f4c671d49af9 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:26:47.862883 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bd424-5157-4388-9b77-f4ae04191d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:26:47.863584 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bd424-5157-4388-9b77-f4ae04191d75 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:26:47.863875 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bd424-5157-4388-9b77-f4ae04191d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:26:47.864159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bd424-5157-4388-9b77-f4ae04191d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:26:47.866009 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-c39bd424-5157-4388-9b77-f4ae04191d75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:26:47.900207 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39bd424-5157-4388-9b77-f4ae04191d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:26:47.901639 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39bd424-5157-4388-9b77-f4ae04191d75 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:26:47.921891 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1451/2902] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:26:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:26:57.732770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=6 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:26:57.919232 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=41 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:11.264867 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee418f-7238-4d91-b7f0-b1c60ddcdf1c service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:11.266291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-acee418f-7238-4d91-b7f0-b1c60ddcdf1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:11.323207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee418f-7238-4d91-b7f0-b1c60ddcdf1c service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:11.326892 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60dc96-23ae-4a49-a435-610f83a288f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80230b1b73944e8ebb6fec3dbe0d5c4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:11.327610 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60dc96-23ae-4a49-a435-610f83a288f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:11.327894 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60dc96-23ae-4a49-a435-610f83a288f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:11.328171 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60dc96-23ae-4a49-a435-610f83a288f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:11.329956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ad60dc96-23ae-4a49-a435-610f83a288f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:11.341214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60dc96-23ae-4a49-a435-610f83a288f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:11.342686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60dc96-23ae-4a49-a435-610f83a288f5 service cinder] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:11.352206 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1452/2903] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:27:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:27:21.348923 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=26 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:22.328159 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55c0d9-3be3-46d6-bbe2-a4c40b117be7 None None] REQUEST_METHOD: `POST` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:22.328748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55c0d9-3be3-46d6-bbe2-a4c40b117be7 None None] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:22.328748 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55c0d9-3be3-46d6-bbe2-a4c40b117be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:22.348859 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.auth.core [None req-3a55c0d9-3be3-46d6-bbe2-a4c40b117be7 None None] MFA Rules not processed for user `d942201423da427b848dfd44b33358b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80183) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:27:22.350076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-3a55c0d9-3be3-46d6-bbe2-a4c40b117be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:22.358163 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1452/2904] 10.222.0.75 () {58 vars in 957 bytes} [Thu Jul 14 11:27:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:27:22.710586 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60dc96-23ae-4a49-a435-610f83a288f5 service cinder] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:22.720524 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60dc96-23ae-4a49-a435-610f83a288f5 service cinder] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:22.725573 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa6d35-638e-43da-aa84-7a8aa7741f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:22.726239 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa6d35-638e-43da-aa84-7a8aa7741f57 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:22.726528 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa6d35-638e-43da-aa84-7a8aa7741f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:22.726814 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa6d35-638e-43da-aa84-7a8aa7741f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:22.728887 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-fdaa6d35-638e-43da-aa84-7a8aa7741f57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:22.739756 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa6d35-638e-43da-aa84-7a8aa7741f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:22.741188 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa6d35-638e-43da-aa84-7a8aa7741f57 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:22.750291 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1453/2905] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:27:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:27:32.341423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:32.746214 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=6 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:38.125900 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55c0d9-3be3-46d6-bbe2-a4c40b117be7 None None] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:38.138947 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55c0d9-3be3-46d6-bbe2-a4c40b117be7 None None] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:38.142816 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f902e-a034-45e2-8b42-df767aea2fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:38.143526 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f902e-a034-45e2-8b42-df767aea2fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:38.143814 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f902e-a034-45e2-8b42-df767aea2fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:38.144122 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f902e-a034-45e2-8b42-df767aea2fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:38.146036 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-222f902e-a034-45e2-8b42-df767aea2fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:38.162835 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f902e-a034-45e2-8b42-df767aea2fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:38.164709 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f902e-a034-45e2-8b42-df767aea2fe3 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:38.178604 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1453/2906] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:27:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:27:41.758267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa6d35-638e-43da-aa84-7a8aa7741f57 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:41.768186 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa6d35-638e-43da-aa84-7a8aa7741f57 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:41.773267 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81a2d2-5ae6-45c1-beca-40dca6c94c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:41.773921 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81a2d2-5ae6-45c1-beca-40dca6c94c55 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:41.774207 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81a2d2-5ae6-45c1-beca-40dca6c94c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:41.774495 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81a2d2-5ae6-45c1-beca-40dca6c94c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:41.776379 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-ea81a2d2-5ae6-45c1-beca-40dca6c94c55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:41.790886 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81a2d2-5ae6-45c1-beca-40dca6c94c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:41.792391 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81a2d2-5ae6-45c1-beca-40dca6c94c55 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:41.805060 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1454/2907] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:27:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:27:46.882140 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f902e-a034-45e2-8b42-df767aea2fe3 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:27:46.892055 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f902e-a034-45e2-8b42-df767aea2fe3 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:27:46.896333 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192b386-4719-4588-8378-426f86e4abc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:27:46.897647 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192b386-4719-4588-8378-426f86e4abc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:27:46.897939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192b386-4719-4588-8378-426f86e4abc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:27:46.898223 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192b386-4719-4588-8378-426f86e4abc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:27:46.900227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-b192b386-4719-4588-8378-426f86e4abc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:27:46.963685 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192b386-4719-4588-8378-426f86e4abc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:27:46.965136 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192b386-4719-4588-8378-426f86e4abc8 service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:27:46.975170 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1454/2908] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:27:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:27:51.799323 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=6 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:27:56.970646 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=36 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:28:47.879781 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81a2d2-5ae6-45c1-beca-40dca6c94c55 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:28:47.889124 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81a2d2-5ae6-45c1-beca-40dca6c94c55 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:28:47.892704 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e33711-997f-417d-8bed-ce20c40dd332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:28:47.893339 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e33711-997f-417d-8bed-ce20c40dd332 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:47.893621 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e33711-997f-417d-8bed-ce20c40dd332 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:47.893896 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e33711-997f-417d-8bed-ce20c40dd332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:47.902134 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e33711-997f-417d-8bed-ce20c40dd332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:28:47.903596 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e33711-997f-417d-8bed-ce20c40dd332 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:28:47.912643 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1455/2909] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:28:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:28:48.789618 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192b386-4719-4588-8378-426f86e4abc8 service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:28:48.809485 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192b386-4719-4588-8378-426f86e4abc8 service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:28:48.817655 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c87ef1-9d32-4e88-82d3-c7747c012233 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e572bf785284a8693cf386ced004524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:28:48.818327 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c87ef1-9d32-4e88-82d3-c7747c012233 service nova] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:48.818617 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c87ef1-9d32-4e88-82d3-c7747c012233 service nova] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:48.818898 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c87ef1-9d32-4e88-82d3-c7747c012233 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:48.821339 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-40c87ef1-9d32-4e88-82d3-c7747c012233 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:48.835553 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c87ef1-9d32-4e88-82d3-c7747c012233 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:28:48.838219 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c87ef1-9d32-4e88-82d3-c7747c012233 service nova] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:28:48.850423 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1455/2910] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:28:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:28:55.016139 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e33711-997f-417d-8bed-ce20c40dd332 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:28:55.026081 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e33711-997f-417d-8bed-ce20c40dd332 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:28:55.030015 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9e28-f74b-4b3a-a98d-d47e5d945d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:28:55.030674 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9e28-f74b-4b3a-a98d-d47e5d945d45 service neutron] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:55.030964 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9e28-f74b-4b3a-a98d-d47e5d945d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:55.031248 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9e28-f74b-4b3a-a98d-d47e5d945d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:55.122447 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf9e28-f74b-4b3a-a98d-d47e5d945d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:28:55.123985 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf9e28-f74b-4b3a-a98d-d47e5d945d45 service neutron] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:28:55.146816 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1456/2911] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:28:58.429938 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c87ef1-9d32-4e88-82d3-c7747c012233 service nova] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:28:58.446135 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c87ef1-9d32-4e88-82d3-c7747c012233 service nova] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:28:58.451367 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f1742-9bc3-478c-906a-e5d80266031f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:28:58.452082 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f1742-9bc3-478c-906a-e5d80266031f service placement] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:28:58.452378 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f1742-9bc3-478c-906a-e5d80266031f service placement] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:28:58.452662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f1742-9bc3-478c-906a-e5d80266031f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:28:58.454465 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-482f1742-9bc3-478c-906a-e5d80266031f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:28:58.469860 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f1742-9bc3-478c-906a-e5d80266031f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:28:58.471785 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f1742-9bc3-478c-906a-e5d80266031f service placement] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:28:58.486865 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1456/2912] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:28:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:29:01.909147 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9e28-f74b-4b3a-a98d-d47e5d945d45 service neutron] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:01.924223 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9e28-f74b-4b3a-a98d-d47e5d945d45 service neutron] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:01.930066 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eb868-491b-4883-99fc-d6529c2d21cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:01.930817 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eb868-491b-4883-99fc-d6529c2d21cf service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:01.931228 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eb868-491b-4883-99fc-d6529c2d21cf service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:01.931704 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eb868-491b-4883-99fc-d6529c2d21cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:01.933860 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-c69eb868-491b-4883-99fc-d6529c2d21cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:01.947434 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69eb868-491b-4883-99fc-d6529c2d21cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:01.949210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69eb868-491b-4883-99fc-d6529c2d21cf service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:01.968980 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1457/2913] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:29:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:29:01.976582 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f1742-9bc3-478c-906a-e5d80266031f service placement] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:01.993051 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f1742-9bc3-478c-906a-e5d80266031f service placement] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:01.998834 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:01.999763 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:02.000053 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:02.000334 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] PATH_INFO: `/v3/users/65c3d05f3b904c0f999984dda155f1cf` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:02.001881 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65c3d05f3b904c0f999984dda155f1cf)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:02.004057 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:02.062734 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65c3d05f3b904c0f999984dda155f1cf'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.063545 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65c3d05f3b904c0f999984dda155f1cf'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.072731 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65c3d05f3b904c0f999984dda155f1cf'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.080409 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65c3d05f3b904c0f999984dda155f1cf'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.081152 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65c3d05f3b904c0f999984dda155f1cf'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.101054 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1457/2914] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:29:01 2022] DELETE /identity/v3/users/65c3d05f3b904c0f999984dda155f1cf => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:02.308478 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eb868-491b-4883-99fc-d6529c2d21cf service placement] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:02.321605 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eb868-491b-4883-99fc-d6529c2d21cf service placement] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:02.402546 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:02.403463 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:02.403751 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:02.404026 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] PATH_INFO: `/v3/projects/67f0b0b647c442b9a682fd37d756400a` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:02.405242 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=67f0b0b647c442b9a682fd37d756400a)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:02.407159 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:02.459477 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67f0b0b647c442b9a682fd37d756400a', 'request_id': 'req-2317286b-a018-454c-b7c4-6c59eee440bb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.461570 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:02.466691 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67f0b0b647c442b9a682fd37d756400a', 'request_id': 'req-2317286b-a018-454c-b7c4-6c59eee440bb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.475285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67f0b0b647c442b9a682fd37d756400a', 'request_id': 'req-2317286b-a018-454c-b7c4-6c59eee440bb'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.505557 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] The token cache is being invalidate because project 67f0b0b647c442b9a682fd37d756400a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:29:02.505874 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.506582 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.508805 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1458/2915] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:29:02 2022] DELETE /identity/v3/projects/67f0b0b647c442b9a682fd37d756400a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:02.513249 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:02.514939 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:02.529156 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67379c-12b5-4bec-bb38-660c2b15a423 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:02.604386 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:02.605287 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:02.605577 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:02.605855 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] PATH_INFO: `/v3/users/d1cdd12de4cc4c19b0182695935a3135` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:02.607109 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1cdd12de4cc4c19b0182695935a3135)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:02.609148 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:02.664501 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1cdd12de4cc4c19b0182695935a3135'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.665265 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1cdd12de4cc4c19b0182695935a3135'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.666622 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:02.674657 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1cdd12de4cc4c19b0182695935a3135'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.683328 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1cdd12de4cc4c19b0182695935a3135'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.684016 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1cdd12de4cc4c19b0182695935a3135'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:02.703919 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1458/2916] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:29:02 2022] DELETE /identity/v3/users/d1cdd12de4cc4c19b0182695935a3135 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:02.888543 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:02.890027 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:02.903770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317286b-a018-454c-b7c4-6c59eee440bb admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:02.965408 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:02.966270 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:02.966555 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:02.966837 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] PATH_INFO: `/v3/projects/b9459e6ccd784ecfb7f268823adc0b84` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:02.968073 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9459e6ccd784ecfb7f268823adc0b84)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:02.970002 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:03.021637 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9459e6ccd784ecfb7f268823adc0b84', 'request_id': 'req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:03.028879 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9459e6ccd784ecfb7f268823adc0b84', 'request_id': 'req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:03.035210 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9459e6ccd784ecfb7f268823adc0b84', 'request_id': 'req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:03.050686 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] The token cache is being invalidate because project b9459e6ccd784ecfb7f268823adc0b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80182) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:29:03.051067 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:03.052055 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:03.054240 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1459/2917] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:29:02 2022] DELETE /identity/v3/projects/b9459e6ccd784ecfb7f268823adc0b84 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:12.700154 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=12,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:13.046993 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:50.866727 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:50.868166 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:50.926634 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cf54c-2042-4cc3-a7e0-79300114ce1c admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:50.984094 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd54025-24f8-43af-95d2-e5eedb7cd7bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dadd485b7cac42f4aa079cab91973c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:50.984765 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd54025-24f8-43af-95d2-e5eedb7cd7bb service neutron] REQUEST_METHOD: `GET` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:50.985056 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd54025-24f8-43af-95d2-e5eedb7cd7bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:50.985343 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd54025-24f8-43af-95d2-e5eedb7cd7bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:50.987847 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-1dd54025-24f8-43af-95d2-e5eedb7cd7bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:51.057365 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd54025-24f8-43af-95d2-e5eedb7cd7bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:51.059032 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd54025-24f8-43af-95d2-e5eedb7cd7bb service neutron] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:51.069589 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1459/2918] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:29:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:29:57.773505 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:57.775029 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:57.789152 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912b3d1-1405-4ecd-adda-e3991dfd5fcc admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:57.793947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:57.794956 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:57.795310 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:57.795693 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] PATH_INFO: `/v3/users/d942201423da427b848dfd44b33358b8` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:57.796992 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d942201423da427b848dfd44b33358b8)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:57.800057 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:57.851597 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd942201423da427b848dfd44b33358b8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:57.859947 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd942201423da427b848dfd44b33358b8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:57.860705 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd942201423da427b848dfd44b33358b8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:57.868770 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd942201423da427b848dfd44b33358b8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:57.869413 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd942201423da427b848dfd44b33358b8'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:57.886619 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1460/2919] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:29:57 2022] DELETE /identity/v3/users/d942201423da427b848dfd44b33358b8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:58.062076 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd54025-24f8-43af-95d2-e5eedb7cd7bb service neutron] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:58.072497 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd54025-24f8-43af-95d2-e5eedb7cd7bb service neutron] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:58.131322 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:58.132216 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:58.132503 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:58.132782 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] PATH_INFO: `/v3/projects/67f41dcdf6564bbd910a2285568941a1` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:58.133980 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67f41dcdf6564bbd910a2285568941a1)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:58.135975 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:58.144422 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=100 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:58.183615 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67f41dcdf6564bbd910a2285568941a1', 'request_id': 'req-baab1162-e9a0-4ac7-bac1-3d124996ae73'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.194144 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67f41dcdf6564bbd910a2285568941a1', 'request_id': 'req-baab1162-e9a0-4ac7-bac1-3d124996ae73'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.204227 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67f41dcdf6564bbd910a2285568941a1', 'request_id': 'req-baab1162-e9a0-4ac7-bac1-3d124996ae73'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.219065 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] The token cache is being invalidate because project 67f41dcdf6564bbd910a2285568941a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:29:58.219474 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.220326 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.222583 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1460/2920] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:29:58 2022] DELETE /identity/v3/projects/67f41dcdf6564bbd910a2285568941a1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:58.226717 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:58.228180 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:58.241285 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3badad-8480-4f4d-ae51-a5ef4774f243 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:58.297219 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:58.298109 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:58.298393 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:58.298677 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] PATH_INFO: `/v3/users/45c0a98b982d444f8aec8107277df32c` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:58.299962 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c0a98b982d444f8aec8107277df32c)` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:58.301897 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:58.347719 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c0a98b982d444f8aec8107277df32c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.355040 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c0a98b982d444f8aec8107277df32c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.355722 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c0a98b982d444f8aec8107277df32c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.362718 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c0a98b982d444f8aec8107277df32c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.363355 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.notifications [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c0a98b982d444f8aec8107277df32c'} {{(pid=80182) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.371009 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:29:58.382359 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1461/2921] 10.222.0.75 () {58 vars in 1223 bytes} [Thu Jul 14 11:29:58 2022] DELETE /identity/v3/users/45c0a98b982d444f8aec8107277df32c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:29:58.565215 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] Authenticating user token {{(pid=80183) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:29:58.566822 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.fernet_utils [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80183) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:29:58.580302 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab1162-e9a0-4ac7-bac1-3d124996ae73 admin admin] Validating token access rules against request {{(pid=80183) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:29:58.639073 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4540918-d183-4636-9c47-59e692e08fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae05ea06648449e9eda17d7c9a3f3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75fd150c50ce43f4ac09652ce03bfda1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80183) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:29:58.640097 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4540918-d183-4636-9c47-59e692e08fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:29:58.640384 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4540918-d183-4636-9c47-59e692e08fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:29:58.640662 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4540918-d183-4636-9c47-59e692e08fe2 admin admin] PATH_INFO: `/v3/projects/b2ef0d38bcd64142b153908b06334091` {{(pid=80183) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:29:58.641844 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4540918-d183-4636-9c47-59e692e08fe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2ef0d38bcd64142b153908b06334091)` {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:29:58.643836 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4540918-d183-4636-9c47-59e692e08fe2 admin admin] RBAC: Authorization granted {{(pid=80183) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:29:58.689826 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e4540918-d183-4636-9c47-59e692e08fe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2ef0d38bcd64142b153908b06334091', 'request_id': 'req-e4540918-d183-4636-9c47-59e692e08fe2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.695812 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e4540918-d183-4636-9c47-59e692e08fe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2ef0d38bcd64142b153908b06334091', 'request_id': 'req-e4540918-d183-4636-9c47-59e692e08fe2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.701246 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e4540918-d183-4636-9c47-59e692e08fe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2ef0d38bcd64142b153908b06334091', 'request_id': 'req-e4540918-d183-4636-9c47-59e692e08fe2'} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.713528 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e4540918-d183-4636-9c47-59e692e08fe2 admin admin] The token cache is being invalidate because project b2ef0d38bcd64142b153908b06334091 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80183) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:29:58.713811 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e4540918-d183-4636-9c47-59e692e08fe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.714437 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG keystone.notifications [None req-e4540918-d183-4636-9c47-59e692e08fe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80183) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:29:58.715999 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: [pid: 80183|app: 0|req: 1461/2922] 10.222.0.75 () {58 vars in 1232 bytes} [Thu Jul 14 11:29:58 2022] DELETE /identity/v3/projects/b2ef0d38bcd64142b153908b06334091 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:30:08.380713 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:30:08.710268 dv-cinder-848938-2-iscsi devstack@keystone.service[80183]: DEBUG dbcounter [-] [80183] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80183) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:31:39.440394 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] Authenticating user token {{(pid=80182) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:31:39.441857 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:31:39.511909 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5216f8-4c69-4ee5-a0be-f0e19bb318d8 admin admin] Validating token access rules against request {{(pid=80182) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:31:39.571321 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5693ca3-04a1-4b07-8ed6-5e4846873c74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fc1f2abce4845219741f4f48bf32d36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68601f16d1c49548b6e3af0b9819ba3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80182) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:31:39.572007 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5693ca3-04a1-4b07-8ed6-5e4846873c74 service placement] REQUEST_METHOD: `GET` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:31:39.572293 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5693ca3-04a1-4b07-8ed6-5e4846873c74 service placement] SCRIPT_NAME: `/identity` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:31:39.572572 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5693ca3-04a1-4b07-8ed6-5e4846873c74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80182) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:31:39.574362 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.fernet_utils [None req-a5693ca3-04a1-4b07-8ed6-5e4846873c74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80182) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:31:39.585749 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5693ca3-04a1-4b07-8ed6-5e4846873c74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:31:39.587337 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5693ca3-04a1-4b07-8ed6-5e4846873c74 service placement] RBAC: Authorization granted {{(pid=80182) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:31:39.598054 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: [pid: 80182|app: 0|req: 1462/2923] 10.222.0.75 () {60 vars in 1365 bytes} [Thu Jul 14 11:31:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:31:49.593064 dv-cinder-848938-2-iscsi devstack@keystone.service[80182]: DEBUG dbcounter [-] [80182] Writing DB stats keystone:SELECT=58 {{(pid=80182) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}